May 25 13:59:52.029283 ip-10-0-128-220 systemd[1]: Starting Kubernetes Kubelet...
May 25 13:59:52.544551 ip-10-0-128-220 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 25 13:59:52.544551 ip-10-0-128-220 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 25 13:59:52.544551 ip-10-0-128-220 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 25 13:59:52.544551 ip-10-0-128-220 kubenswrapper[2571]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 25 13:59:52.544551 ip-10-0-128-220 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 25 13:59:52.546952 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.546383    2571 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 25 13:59:52.555402 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555355    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 25 13:59:52.555402 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555390    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 25 13:59:52.555402 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555395    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 25 13:59:52.555402 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555400    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 25 13:59:52.555402 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555405    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 25 13:59:52.555402 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555409    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 25 13:59:52.555402 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555412    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 25 13:59:52.555402 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555415    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 25 13:59:52.555402 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555418    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 25 13:59:52.555402 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555421    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 25 13:59:52.555402 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555424    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 25 13:59:52.555402 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555427    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 25 13:59:52.555903 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555431    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 25 13:59:52.555903 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555434    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 25 13:59:52.555903 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555436    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 25 13:59:52.555903 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555439    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 25 13:59:52.555903 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555441    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 25 13:59:52.555903 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555444    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 25 13:59:52.555903 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555447    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 25 13:59:52.555903 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555449    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 25 13:59:52.555903 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555451    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 25 13:59:52.555903 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555454    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 25 13:59:52.555903 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555457    2571 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 25 13:59:52.555903 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555459    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 25 13:59:52.555903 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555462    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 25 13:59:52.555903 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555465    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 25 13:59:52.555903 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555467    2571 feature_gate.go:328] unrecognized feature gate: Example
May 25 13:59:52.555903 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555470    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 25 13:59:52.555903 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555473    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 25 13:59:52.555903 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555475    2571 feature_gate.go:328] unrecognized feature gate: Example2
May 25 13:59:52.555903 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555477    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 25 13:59:52.555903 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555480    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 25 13:59:52.556454 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555483    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 25 13:59:52.556454 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555485    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 25 13:59:52.556454 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555488    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 25 13:59:52.556454 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555491    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 25 13:59:52.556454 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555495    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 25 13:59:52.556454 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555499    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 25 13:59:52.556454 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555503    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 25 13:59:52.556454 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555506    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 25 13:59:52.556454 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555508    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 25 13:59:52.556454 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555511    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
May 25 13:59:52.556454 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555513    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 25 13:59:52.556454 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555516    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 25 13:59:52.556454 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555518    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 25 13:59:52.556454 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555522    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 25 13:59:52.556454 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555525    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 25 13:59:52.556454 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555527    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 25 13:59:52.556454 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555531    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 25 13:59:52.556454 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555534    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 25 13:59:52.556454 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555536    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 25 13:59:52.556961 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555539    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 25 13:59:52.556961 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555542    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 25 13:59:52.556961 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555544    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 25 13:59:52.556961 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555548    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
May 25 13:59:52.556961 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555551    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 25 13:59:52.556961 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555553    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 25 13:59:52.556961 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555555    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 25 13:59:52.556961 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555559    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 25 13:59:52.556961 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555562    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 25 13:59:52.556961 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555564    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 25 13:59:52.556961 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555567    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 25 13:59:52.556961 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555570    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 25 13:59:52.556961 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555572    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 25 13:59:52.556961 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555575    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 25 13:59:52.556961 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555577    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 25 13:59:52.556961 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555580    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 25 13:59:52.556961 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555582    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 25 13:59:52.556961 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555585    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 25 13:59:52.556961 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555588    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 25 13:59:52.556961 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555590    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 25 13:59:52.557453 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555593    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 25 13:59:52.557453 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555595    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 25 13:59:52.557453 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555598    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 25 13:59:52.557453 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555600    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 25 13:59:52.557453 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555603    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 25 13:59:52.557453 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555605    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 25 13:59:52.557453 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555611    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 25 13:59:52.557453 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555616    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 25 13:59:52.557453 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555619    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 25 13:59:52.557453 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555621    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 25 13:59:52.557453 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555625    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 25 13:59:52.557453 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555627    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 25 13:59:52.557453 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555630    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 25 13:59:52.557453 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555632    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 25 13:59:52.557453 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.555635    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 25 13:59:52.557837 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557667    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 25 13:59:52.557837 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557681    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 25 13:59:52.557837 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557684    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 25 13:59:52.557837 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557688    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 25 13:59:52.557837 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557692    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 25 13:59:52.557837 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557695    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 25 13:59:52.557837 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557698    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 25 13:59:52.557837 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557701    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 25 13:59:52.557837 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557703    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 25 13:59:52.557837 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557706    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 25 13:59:52.557837 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557709    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 25 13:59:52.557837 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557711    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 25 13:59:52.557837 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557713    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 25 13:59:52.557837 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557716    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 25 13:59:52.557837 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557719    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
May 25 13:59:52.557837 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557722    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 25 13:59:52.557837 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557725    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 25 13:59:52.557837 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557727    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 25 13:59:52.557837 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557730    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 25 13:59:52.557837 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557733    2571 feature_gate.go:328] unrecognized feature gate: Example
May 25 13:59:52.558366 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557736    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 25 13:59:52.558366 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557738    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 25 13:59:52.558366 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557740    2571 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 25 13:59:52.558366 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557743    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 25 13:59:52.558366 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557747    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 25 13:59:52.558366 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557750    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 25 13:59:52.558366 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557752    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 25 13:59:52.558366 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557755    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 25 13:59:52.558366 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557757    2571 feature_gate.go:328] unrecognized feature gate: Example2
May 25 13:59:52.558366 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557760    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 25 13:59:52.558366 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557762    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 25 13:59:52.558366 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557764    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 25 13:59:52.558366 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557767    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 25 13:59:52.558366 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557771    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 25 13:59:52.558366 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557774    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 25 13:59:52.558366 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557777    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 25 13:59:52.558366 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557779    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 25 13:59:52.558366 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557782    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 25 13:59:52.558366 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557784    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 25 13:59:52.558366 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557787    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 25 13:59:52.558965 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557789    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 25 13:59:52.558965 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557792    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
May 25 13:59:52.558965 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557795    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 25 13:59:52.558965 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557797    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 25 13:59:52.558965 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557800    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 25 13:59:52.558965 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557803    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 25 13:59:52.558965 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557807    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 25 13:59:52.558965 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557810    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 25 13:59:52.558965 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557812    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 25 13:59:52.558965 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557814    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 25 13:59:52.558965 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557817    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 25 13:59:52.558965 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557820    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 25 13:59:52.558965 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557822    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 25 13:59:52.558965 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557825    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 25 13:59:52.558965 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557827    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 25 13:59:52.558965 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557830    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 25 13:59:52.558965 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557832    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 25 13:59:52.558965 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557844    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 25 13:59:52.558965 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557847    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 25 13:59:52.558965 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557850    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 25 13:59:52.559478 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557853    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 25 13:59:52.559478 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557857    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 25 13:59:52.559478 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557861    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 25 13:59:52.559478 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557865    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 25 13:59:52.559478 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557868    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 25 13:59:52.559478 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557872    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 25 13:59:52.559478 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557876    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 25 13:59:52.559478 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557881    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 25 13:59:52.559478 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557884    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 25 13:59:52.559478 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557888    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 25 13:59:52.559478 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557892    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 25 13:59:52.559478 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557895    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 25 13:59:52.559478 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557898    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 25 13:59:52.559478 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557902    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 25 13:59:52.559478 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557904    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 25 13:59:52.559478 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557907    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 25 13:59:52.559478 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557910    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 25 13:59:52.559478 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557912    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 25 13:59:52.559478 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557915    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 25 13:59:52.559989 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557917    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 25 13:59:52.559989 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557920    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 25 13:59:52.559989 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557922    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 25 13:59:52.559989 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557925    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 25 13:59:52.559989 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557927    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 25 13:59:52.559989 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557950    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 25 13:59:52.559989 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.557953    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 25 13:59:52.559989 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558053    2571 flags.go:64] FLAG: --address="0.0.0.0"
May 25 13:59:52.559989 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558067    2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 25 13:59:52.559989 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558077    2571 flags.go:64] FLAG: --anonymous-auth="true"
May 25 13:59:52.559989 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558083    2571 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 25 13:59:52.559989 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558098    2571 flags.go:64] FLAG: --authentication-token-webhook="false"
May 25 13:59:52.559989 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558102    2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 25 13:59:52.559989 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558108    2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 25 13:59:52.559989 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558113    2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 25 13:59:52.559989 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558116    2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 25 13:59:52.559989 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558121    2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 25 13:59:52.559989 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558127    2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 25 13:59:52.559989 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558132    2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 25 13:59:52.559989 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558137    2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 25 13:59:52.559989 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558141    2571 flags.go:64] FLAG: --cgroup-root=""
May 25 13:59:52.559989 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558144    2571 flags.go:64] FLAG: --cgroups-per-qos="true"
May 25 13:59:52.560569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558148    2571 flags.go:64] FLAG: --client-ca-file=""
May 25 13:59:52.560569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558151    2571 flags.go:64] FLAG: --cloud-config=""
May 25 13:59:52.560569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558153    2571 flags.go:64] FLAG: --cloud-provider="external"
May 25 13:59:52.560569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558157    2571 flags.go:64] FLAG: --cluster-dns="[]"
May 25 13:59:52.560569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558166    2571 flags.go:64] FLAG: --cluster-domain=""
May 25 13:59:52.560569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558169    2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 25 13:59:52.560569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558172    2571 flags.go:64] FLAG: --config-dir=""
May 25 13:59:52.560569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558175    2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 25 13:59:52.560569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558179    2571 flags.go:64] FLAG: --container-log-max-files="5"
May 25 13:59:52.560569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558184    2571 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 25 13:59:52.560569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558187    2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 25 13:59:52.560569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558191    2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 25 13:59:52.560569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558195    2571 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 25 13:59:52.560569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558199    2571 flags.go:64] FLAG: --contention-profiling="false"
May 25 13:59:52.560569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558202    2571 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 25 13:59:52.560569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558205    2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 25 13:59:52.560569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558208    2571 flags.go:64] FLAG: --cpu-manager-policy="none"
May 25 13:59:52.560569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558212    2571 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 25 13:59:52.560569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558217    2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 25 13:59:52.560569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558220    2571 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 25 13:59:52.560569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558224    2571 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 25 13:59:52.560569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558227    2571 flags.go:64] FLAG: --enable-load-reader="false"
May 25 13:59:52.560569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558231    2571 flags.go:64] FLAG: --enable-server="true"
May 25 13:59:52.560569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558234    2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 25 13:59:52.560569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558239    2571 flags.go:64] FLAG: --event-burst="100"
May 25 13:59:52.561216 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558242    2571 flags.go:64] FLAG: --event-qps="50"
May 25 13:59:52.561216 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558245    2571 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 25 13:59:52.561216 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558248    2571 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 25 13:59:52.561216 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558255    2571 flags.go:64] FLAG: --eviction-hard=""
May 25 13:59:52.561216 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558259    2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 25 13:59:52.561216 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558262    2571 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 25 13:59:52.561216 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558265    2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 25 13:59:52.561216 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558268    2571 flags.go:64] FLAG: --eviction-soft=""
May 25 13:59:52.561216 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558272    2571 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 25 13:59:52.561216 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558275    2571 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 25 13:59:52.561216 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558278    2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 25 13:59:52.561216 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558281    2571 flags.go:64] FLAG: --experimental-mounter-path=""
May 25 13:59:52.561216 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558283    2571 flags.go:64] FLAG: --fail-cgroupv1="false"
May 25 13:59:52.561216 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558286    2571 flags.go:64] FLAG: --fail-swap-on="true"
May 25 13:59:52.561216 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558289    2571 flags.go:64] FLAG: --feature-gates=""
May 25 13:59:52.561216 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558294    2571 flags.go:64] FLAG: --file-check-frequency="20s"
May 25 13:59:52.561216 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558297    2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 25 13:59:52.561216 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558300    2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 25 13:59:52.561216 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558303    2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 25 13:59:52.561216 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558307    2571 flags.go:64] FLAG: --healthz-port="10248"
May 25 13:59:52.561216 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558310    2571 flags.go:64] FLAG: --help="false"
May 25 13:59:52.561216 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558313    2571 flags.go:64] FLAG: --hostname-override="ip-10-0-128-220.ec2.internal"
May 25 13:59:52.561216 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558316    2571 flags.go:64] FLAG: --housekeeping-interval="10s"
May 25 13:59:52.561216 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558319    2571 flags.go:64] FLAG: --http-check-frequency="20s"
May 25 13:59:52.561815 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558322    2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 25 13:59:52.561815 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558326    2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 25 13:59:52.561815 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558330    2571 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 25 13:59:52.561815 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558333    2571 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 25 13:59:52.561815 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558336    2571 flags.go:64] FLAG: --image-service-endpoint=""
May 25 13:59:52.561815 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558339    2571 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 25 13:59:52.561815 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558343    2571 flags.go:64] FLAG: --kube-api-burst="100"
May 25 13:59:52.561815 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558346    2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 25 13:59:52.561815 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558350    2571 flags.go:64] FLAG: --kube-api-qps="50"
May 25 13:59:52.561815 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558352    2571 flags.go:64] FLAG: --kube-reserved=""
May 25 13:59:52.561815 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558355    2571 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 25 13:59:52.561815 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558360    2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 25 13:59:52.561815 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558364    2571 flags.go:64] FLAG: --kubelet-cgroups=""
May 25 13:59:52.561815 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558367    2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 25 13:59:52.561815 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558370    2571 flags.go:64] FLAG: --lock-file=""
May 25 13:59:52.561815 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558373    2571 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 25 13:59:52.561815 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558377    2571 flags.go:64] FLAG: --log-flush-frequency="5s"
May 25 13:59:52.561815 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558381    2571 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 25 13:59:52.561815 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558388    2571 flags.go:64] FLAG: --log-json-split-stream="false"
May 25 13:59:52.561815 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558391    2571 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 25 13:59:52.561815 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558394    2571 flags.go:64] FLAG: --log-text-split-stream="false"
May 25 13:59:52.561815 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558397    2571 flags.go:64] FLAG: --logging-format="text"
May 25 13:59:52.561815 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558400    2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 25 13:59:52.562410 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558408    2571 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 25 13:59:52.562410 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558411    2571 flags.go:64] FLAG: --manifest-url=""
May 25 13:59:52.562410 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558414    2571 flags.go:64] FLAG: --manifest-url-header=""
May 25 13:59:52.562410 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558420    2571 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 25 13:59:52.562410 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558423    2571 flags.go:64] FLAG: --max-open-files="1000000"
May 25 13:59:52.562410 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558428    2571 flags.go:64] FLAG: --max-pods="110"
May 25 13:59:52.562410 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558431    2571 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 25 13:59:52.562410 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558434    2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 25 13:59:52.562410 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558437    2571 flags.go:64] FLAG: --memory-manager-policy="None"
May 25 13:59:52.562410 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558440    2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 25 13:59:52.562410 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558444    2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 25 13:59:52.562410 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558447    2571 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 25 13:59:52.562410 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558450    2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 25 13:59:52.562410 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558460    2571 flags.go:64] FLAG: --node-status-max-images="50"
May 25 13:59:52.562410 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558463    2571 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 25 13:59:52.562410 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558466    2571 flags.go:64] FLAG: --oom-score-adj="-999"
May 25 13:59:52.562410 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558470    2571 flags.go:64] FLAG: --pod-cidr=""
May 25 13:59:52.562410 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558473    2571 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc"
May 25 13:59:52.562410 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558481    2571 flags.go:64] FLAG: --pod-manifest-path=""
May 25 13:59:52.562410 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558485    2571 flags.go:64] FLAG: --pod-max-pids="-1"
May 25 13:59:52.562410 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558490    2571 flags.go:64] FLAG: --pods-per-core="0"
May 25 13:59:52.562410 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558493    2571 flags.go:64] FLAG: --port="10250"
May 25 13:59:52.562410 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558496    2571 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 25 13:59:52.562410 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558499    2571 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08d8fbba2cde4ff60"
May 25 13:59:52.563030 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558503    2571 flags.go:64] FLAG: --qos-reserved=""
May 25 13:59:52.563030 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558506    2571 flags.go:64] FLAG: --read-only-port="10255"
May 25 13:59:52.563030 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558509    2571 flags.go:64] FLAG: --register-node="true"
May 25 13:59:52.563030 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558512    2571 flags.go:64] FLAG: --register-schedulable="true"
May 25 13:59:52.563030 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558515    2571 flags.go:64] FLAG: --register-with-taints=""
May 25 13:59:52.563030 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558519    2571 flags.go:64] FLAG: --registry-burst="10"
May 25 13:59:52.563030 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558522    2571 flags.go:64] FLAG: --registry-qps="5"
May 25 13:59:52.563030 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558525    2571 flags.go:64] FLAG: --reserved-cpus=""
May 25 13:59:52.563030 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558528    2571 flags.go:64] FLAG: --reserved-memory=""
May 25 13:59:52.563030 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558534    2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 25 13:59:52.563030 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558537    2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 25 13:59:52.563030 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558541    2571 flags.go:64] FLAG: --rotate-certificates="false"
May 25 13:59:52.563030 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558544    2571 flags.go:64] FLAG: --rotate-server-certificates="false"
May 25 13:59:52.563030 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558547    2571 flags.go:64] FLAG: --runonce="false"
May 25 13:59:52.563030 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558550    2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 25 13:59:52.563030 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558553    2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 25 13:59:52.563030 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558556    2571 flags.go:64] FLAG: --seccomp-default="false"
May 25 13:59:52.563030 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558559    2571 flags.go:64] FLAG: --serialize-image-pulls="true"
May 25 13:59:52.563030 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558562    2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 25 13:59:52.563030 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558565    2571 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 25 13:59:52.563030 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558569    2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 25 13:59:52.563030 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558572    2571 flags.go:64] FLAG: --storage-driver-password="root"
May 25 13:59:52.563030 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558575    2571 flags.go:64] FLAG: --storage-driver-secure="false"
May 25 13:59:52.563030 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558578    2571 flags.go:64] FLAG: --storage-driver-table="stats"
May 25 13:59:52.563030 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558582    2571 flags.go:64] FLAG: --storage-driver-user="root"
May 25 13:59:52.563030 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558585    2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 25 13:59:52.563797 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558588    2571 flags.go:64] FLAG: --sync-frequency="1m0s"
May 25 13:59:52.563797 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558591    2571 flags.go:64] FLAG: --system-cgroups=""
May 25 13:59:52.563797 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558601    2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 25 13:59:52.563797 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558607    2571 flags.go:64] FLAG: --system-reserved-cgroup=""
May 25 13:59:52.563797 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558610    2571 flags.go:64] FLAG: --tls-cert-file=""
May 25 13:59:52.563797 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558613    2571 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 25 13:59:52.563797 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558618    2571 flags.go:64] FLAG: --tls-min-version=""
May 25 13:59:52.563797 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558621    2571 flags.go:64] FLAG: --tls-private-key-file=""
May 25 13:59:52.563797 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558624    2571 flags.go:64] FLAG: --topology-manager-policy="none"
May 25 13:59:52.563797 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558627    2571 flags.go:64] FLAG: --topology-manager-policy-options=""
May 25 13:59:52.563797 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558630    2571 flags.go:64] FLAG: --topology-manager-scope="container"
May 25 13:59:52.563797 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558633    2571 flags.go:64] FLAG: --v="2"
May 25 13:59:52.563797 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558638    2571 flags.go:64] FLAG: --version="false"
May 25 13:59:52.563797 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558643    2571 flags.go:64] FLAG: --vmodule=""
May 25 13:59:52.563797 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558648    2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 25 13:59:52.563797 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.558652    2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 25 13:59:52.563797 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558770    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 25 13:59:52.563797 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558775    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 25 13:59:52.563797 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558778    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 25 13:59:52.563797 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558782    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 25 13:59:52.563797 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558785    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 25 13:59:52.563797 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558789    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 25 13:59:52.563797 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558792    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 25 13:59:52.564457 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558795    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 25 13:59:52.564457 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558798    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 25 13:59:52.564457 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558801    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 25 13:59:52.564457 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558804    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 25 13:59:52.564457 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558807    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 25 13:59:52.564457 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558810    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 25 13:59:52.564457 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558813    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 25 13:59:52.564457 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558815    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
May 25 13:59:52.564457 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558818    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 25 13:59:52.564457 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558821    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 25 13:59:52.564457 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558823    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 25 13:59:52.564457 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558828    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 25 13:59:52.564457 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558831    2571 feature_gate.go:328] unrecognized feature gate: Example
May 25 13:59:52.564457 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558834    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 25 13:59:52.564457 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558836    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 25 13:59:52.564457 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558839    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 25 13:59:52.564457 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558843    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 25 13:59:52.564457 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558846    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 25 13:59:52.564457 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558849    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 25 13:59:52.564457 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558852    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 25 13:59:52.565026 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558855    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 25 13:59:52.565026 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558857    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 25 13:59:52.565026 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558861    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 25 13:59:52.565026 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558863    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 25 13:59:52.565026 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558868    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 25 13:59:52.565026 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558870    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 25 13:59:52.565026 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558873    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 25 13:59:52.565026 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558877    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 25 13:59:52.565026 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558879    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 25 13:59:52.565026 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558882    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 25 13:59:52.565026 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558885    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
May 25 13:59:52.565026 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558888    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 25 13:59:52.565026 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558891    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 25 13:59:52.565026 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558893    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 25 13:59:52.565026 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558896    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 25 13:59:52.565026 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558898    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 25 13:59:52.565026 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558901    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 25 13:59:52.565026 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558904    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 25 13:59:52.565026 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558908    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 25 13:59:52.565026 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558910    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 25 13:59:52.565545 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558914    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 25 13:59:52.565545 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558916    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 25 13:59:52.565545 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558920    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 25 13:59:52.565545 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558924    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 25 13:59:52.565545 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558927    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 25 13:59:52.565545 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558955    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 25 13:59:52.565545 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558960    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 25 13:59:52.565545 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558964    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 25 13:59:52.565545 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558968    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 25 13:59:52.565545 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558972    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 25 13:59:52.565545 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558975    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 25 13:59:52.565545 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558978    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 25 13:59:52.565545 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558981    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 25 13:59:52.565545 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558984    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 25 13:59:52.565545 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558988    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 25 13:59:52.565545 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558990    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 25 13:59:52.565545 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.558996    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 25 13:59:52.565545 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.559000    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 25 13:59:52.565545 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.559003    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 25 13:59:52.566053 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.559006    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 25 13:59:52.566053 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.559009    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 25 13:59:52.566053 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.559012    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 25 13:59:52.566053 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.559014    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 25 13:59:52.566053 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.559016    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 25 13:59:52.566053 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.559019    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 25 13:59:52.566053 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.559021    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 25 13:59:52.566053 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.559024    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 25 13:59:52.566053 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.559027    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 25 13:59:52.566053 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.559030    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 25 13:59:52.566053 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.559032    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 25 13:59:52.566053 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.559034    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 25 13:59:52.566053 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.559037    2571 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 25 13:59:52.566053 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.559040    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 25 13:59:52.566053 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.559043    2571 feature_gate.go:328] unrecognized feature gate: Example2
May 25 13:59:52.566053 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.559045    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 25 13:59:52.566053 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.559052    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 25 13:59:52.566053 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.559055    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 25 13:59:52.566053 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.559057    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 25 13:59:52.566539 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.559060    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 25 13:59:52.566539 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.559066    2571 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 25 13:59:52.570110 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.570069    2571 server.go:530] "Kubelet version" kubeletVersion="v1.33.11"
May 25 13:59:52.570110 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.570108    2571 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 25 13:59:52.570271 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570172    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 25 13:59:52.570271 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570183    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 25 13:59:52.570271 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570186    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 25 13:59:52.570271 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570189    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 25 13:59:52.570271 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570192    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 25 13:59:52.570271 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570195    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 25 13:59:52.570271 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570198    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 25 13:59:52.570271 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570201    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 25 13:59:52.570271 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570203    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 25 13:59:52.570271 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570206    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 25 13:59:52.570271 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570210    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 25 13:59:52.570271 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570213    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 25 13:59:52.570271 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570215    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 25 13:59:52.570271 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570218    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 25 13:59:52.570271 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570221    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 25 13:59:52.570271 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570225    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 25 13:59:52.570271 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570230    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 25 13:59:52.570271 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570233    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 25 13:59:52.570271 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570236    2571 feature_gate.go:328] unrecognized feature gate: Example2
May 25 13:59:52.570743 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570239    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 25 13:59:52.570743 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570241    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 25 13:59:52.570743 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570244    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 25 13:59:52.570743 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570247    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 25 13:59:52.570743 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570250    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
May 25 13:59:52.570743 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570252    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 25 13:59:52.570743 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570255    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 25 13:59:52.570743 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570257    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 25 13:59:52.570743 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570260    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 25 13:59:52.570743 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570262    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 25 13:59:52.570743 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570265    2571 feature_gate.go:328] unrecognized feature gate: Example
May 25 13:59:52.570743 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570268    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 25 13:59:52.570743 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570272    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 25 13:59:52.570743 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570275    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 25 13:59:52.570743 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570278    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 25 13:59:52.570743 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570281    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 25 13:59:52.570743 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570284    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 25 13:59:52.570743 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570287    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 25 13:59:52.570743 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570289    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 25 13:59:52.571263 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570292    2571 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 25 13:59:52.571263 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570295    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 25 13:59:52.571263 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570298    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 25 13:59:52.571263 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570302    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 25 13:59:52.571263 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570305    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 25 13:59:52.571263 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570308    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 25 13:59:52.571263 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570311    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 25 13:59:52.571263 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570313    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 25 13:59:52.571263 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570316    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 25 13:59:52.571263 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570318    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 25 13:59:52.571263 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570321    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 25 13:59:52.571263 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570324    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 25 13:59:52.571263 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570326    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 25 13:59:52.571263 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570328    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 25 13:59:52.571263 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570332    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 25 13:59:52.571263 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570336    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 25 13:59:52.571263 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570338    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 25 13:59:52.571263 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570341    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 25 13:59:52.571263 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570344    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 25 13:59:52.571263 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570346    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 25 13:59:52.571782 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570349    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 25 13:59:52.571782 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570352    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 25 13:59:52.571782 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570355    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 25 13:59:52.571782 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570358    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 25 13:59:52.571782 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570360    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 25 13:59:52.571782 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570363    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
May 25 13:59:52.571782 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570366    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 25 13:59:52.571782 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570369    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 25 13:59:52.571782 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570371    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 25 13:59:52.571782 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570374    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 25 13:59:52.571782 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570377    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 25 13:59:52.571782 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570379    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 25 13:59:52.571782 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570382    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 25 13:59:52.571782 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570384    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 25 13:59:52.571782 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570387    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 25 13:59:52.571782 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570390    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 25 13:59:52.571782 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570392    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 25 13:59:52.571782 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570395    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 25 13:59:52.571782 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570397    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 25 13:59:52.571782 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570400    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 25 13:59:52.572525 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570402    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 25 13:59:52.572525 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570405    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 25 13:59:52.572525 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570407    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 25 13:59:52.572525 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570410    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 25 13:59:52.572525 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570412    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 25 13:59:52.572525 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570415    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 25 13:59:52.572525 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570420    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 25 13:59:52.572525 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570423    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 25 13:59:52.572525 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.570429    2571 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 25 13:59:52.572525 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570555    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 25 13:59:52.572525 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570561    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 25 13:59:52.572525 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570564    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 25 13:59:52.572525 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570567    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 25 13:59:52.572525 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570570    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 25 13:59:52.572525 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570573    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 25 13:59:52.572925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570576    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 25 13:59:52.572925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570579    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 25 13:59:52.572925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570582    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 25 13:59:52.572925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570585    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 25 13:59:52.572925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570588    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
May 25 13:59:52.572925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570591    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 25 13:59:52.572925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570594    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 25 13:59:52.572925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570596    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 25 13:59:52.572925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570599    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 25 13:59:52.572925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570601    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 25 13:59:52.572925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570604    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
May 25 13:59:52.572925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570606    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 25 13:59:52.572925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570609    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 25 13:59:52.572925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570611    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 25 13:59:52.572925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570614    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 25 13:59:52.572925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570617    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 25 13:59:52.572925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570619    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 25 13:59:52.572925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570623    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 25 13:59:52.572925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570628    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 25 13:59:52.572925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570631    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 25 13:59:52.573448 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570635    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 25 13:59:52.573448 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570637    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 25 13:59:52.573448 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570640    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 25 13:59:52.573448 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570643    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 25 13:59:52.573448 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570646    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 25 13:59:52.573448 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570649    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 25 13:59:52.573448 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570652    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 25 13:59:52.573448 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570654    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 25 13:59:52.573448 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570657    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 25 13:59:52.573448 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570660    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 25 13:59:52.573448 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570663    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 25 13:59:52.573448 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570666    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 25 13:59:52.573448 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570668    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 25 13:59:52.573448 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570671    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 25 13:59:52.573448 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570674    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 25 13:59:52.573448 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570677    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 25 13:59:52.573448 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570679    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 25 13:59:52.573448 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570684    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 25 13:59:52.573448 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570687    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 25 13:59:52.573925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570690    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 25 13:59:52.573925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570692    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 25 13:59:52.573925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570695    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 25 13:59:52.573925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570698    2571 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 25 13:59:52.573925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570700    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 25 13:59:52.573925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570703    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 25 13:59:52.573925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570705    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 25 13:59:52.573925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570708    2571 feature_gate.go:328] unrecognized feature gate: Example2
May 25 13:59:52.573925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570710    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 25 13:59:52.573925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570713    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 25 13:59:52.573925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570716    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 25 13:59:52.573925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570718    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 25 13:59:52.573925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570721    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 25 13:59:52.573925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570723    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 25 13:59:52.573925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570725    2571 feature_gate.go:328] unrecognized feature gate: Example
May 25 13:59:52.573925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570728    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 25 13:59:52.573925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570732    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 25 13:59:52.573925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570735    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 25 13:59:52.573925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570738    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 25 13:59:52.573925 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570740    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 25 13:59:52.574449 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570743    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 25 13:59:52.574449 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570747    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 25 13:59:52.574449 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570751    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 25 13:59:52.574449 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570754    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 25 13:59:52.574449 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570757    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 25 13:59:52.574449 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570760    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 25 13:59:52.574449 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570763    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 25 13:59:52.574449 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570766    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 25 13:59:52.574449 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570769    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 25 13:59:52.574449 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570772    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 25 13:59:52.574449 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570774    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 25 13:59:52.574449 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570778    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 25 13:59:52.574449 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570781    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 25 13:59:52.574449 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570783    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 25 13:59:52.574449 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570786    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 25 13:59:52.574449 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570789    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 25 13:59:52.574449 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570791    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 25 13:59:52.574449 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570794    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 25 13:59:52.574449 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570797    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 25 13:59:52.575013 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570799    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 25 13:59:52.575013 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:52.570801    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 25 13:59:52.575013 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.570807    2571 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 25 13:59:52.575013 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.571797    2571 server.go:962] "Client rotation is on, will bootstrap in background"
May 25 13:59:52.575013 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.574609    2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 25 13:59:52.575788 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.575767    2571 server.go:1019] "Starting client certificate rotation"
May 25 13:59:52.575913 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.575886    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 25 13:59:52.575976 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.575963    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 25 13:59:52.605680 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.605654    2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 25 13:59:52.606875 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.606853    2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 25 13:59:52.627000 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.626971    2571 log.go:25] "Validated CRI v1 runtime API"
May 25 13:59:52.633137 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.633115    2571 log.go:25] "Validated CRI v1 image API"
May 25 13:59:52.634647 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.634625    2571 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 25 13:59:52.639834 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.639802    2571 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b1badd2f-7e6f-4e80-bc59-4ddaddf11ea8:/dev/nvme0n1p3 e1c73a9d-70c4-4fcd-adaf-6d6613a5209b:/dev/nvme0n1p4]
May 25 13:59:52.639898 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.639835    2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 25 13:59:52.644575 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.644454    2571 manager.go:217] Machine: {Timestamp:2026-05-25 13:59:52.643921838 +0000 UTC m=+0.478777923 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:2996416 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2834af5664843dcb5ff1eb85020b9d SystemUUID:ec2834af-5664-843d-cb5f-f1eb85020b9d BootID:5fc70fa6-fef6-46a8-9ce4-56cdb6da6bcc Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3d:32:53:a6:0f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3d:32:53:a6:0f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8e:39:ef:a0:62:af Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 25 13:59:52.644575 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.644571    2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 25 13:59:52.644718 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.644705    2571 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 25 13:59:52.645808 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.645786    2571 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 25 13:59:52.645877 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.645851    2571 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 25 13:59:52.646051 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.645880    2571 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-220.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 25 13:59:52.646095 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.646062    2571 topology_manager.go:138] "Creating topology manager with none policy"
May 25 13:59:52.646095 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.646071    2571 container_manager_linux.go:306] "Creating device plugin manager"
May 25 13:59:52.646095 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.646079    2571 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 25 13:59:52.646095 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.646095    2571 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 25 13:59:52.648631 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.648619    2571 state_mem.go:36] "Initialized new in-memory state store"
May 25 13:59:52.648748 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.648738    2571 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 25 13:59:52.651341 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.651326    2571 kubelet.go:491] "Attempting to sync node with API server"
May 25 13:59:52.651341 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.651343    2571 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 25 13:59:52.652118 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.652108    2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 25 13:59:52.652158 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.652123    2571 kubelet.go:397] "Adding apiserver pod source"
May 25 13:59:52.652158 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.652132    2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 25 13:59:52.653450 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.653437    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 25 13:59:52.653486 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.653456    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 25 13:59:52.656779 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.656760    2571 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 25 13:59:52.659073 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.659060    2571 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 25 13:59:52.660013 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.660000    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 25 13:59:52.660058 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.660025    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 25 13:59:52.660058 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.660035    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 25 13:59:52.660058 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.660041    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 25 13:59:52.660058 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.660049    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 25 13:59:52.660058 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.660055    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 25 13:59:52.660197 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.660061    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 25 13:59:52.660197 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.660067    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 25 13:59:52.660197 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.660075    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 25 13:59:52.660197 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.660080    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 25 13:59:52.660197 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.660088    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 25 13:59:52.660461 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.660452    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 25 13:59:52.661527 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.661513    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 25 13:59:52.661527 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.661521    2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 25 13:59:52.664879 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.664863    2571 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 25 13:59:52.664879 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.664869    2571 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-220.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 25 13:59:52.665020 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.664920    2571 server.go:1295] "Started kubelet"
May 25 13:59:52.665366 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.665328    2571 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 25 13:59:52.665415 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.665380    2571 server_v1.go:47] "podresources" method="list" useActivePods=true
May 25 13:59:52.665456 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.665383    2571 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 25 13:59:52.665650 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:52.665628    2571 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-220.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 25 13:59:52.665723 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:52.665629    2571 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 25 13:59:52.665668 ip-10-0-128-220 systemd[1]: Started Kubernetes Kubelet.
May 25 13:59:52.670026 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.670002    2571 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 25 13:59:52.672901 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.672880    2571 server.go:317] "Adding debug handlers to kubelet server"
May 25 13:59:52.675701 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:52.674627    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-220.ec2.internal.18b2d3b89f4dad43  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-220.ec2.internal,UID:ip-10-0-128-220.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-220.ec2.internal,},FirstTimestamp:2026-05-25 13:59:52.664878403 +0000 UTC m=+0.499734494,LastTimestamp:2026-05-25 13:59:52.664878403 +0000 UTC m=+0.499734494,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-220.ec2.internal,}"
May 25 13:59:52.680158 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.680140    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 25 13:59:52.680851 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:52.680823    2571 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 25 13:59:52.681315 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.680867    2571 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 25 13:59:52.681315 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.680981    2571 volume_manager.go:295] "The desired_state_of_world populator starts"
May 25 13:59:52.681315 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.680991    2571 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 25 13:59:52.681315 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.681030    2571 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 25 13:59:52.681315 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.681092    2571 reconstruct.go:97] "Volume reconstruction finished"
May 25 13:59:52.681315 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.681099    2571 reconciler.go:26] "Reconciler: start to sync state"
May 25 13:59:52.681581 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.681543    2571 factory.go:55] Registering systemd factory
May 25 13:59:52.681581 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.681561    2571 factory.go:223] Registration of the systemd container factory successfully
May 25 13:59:52.682009 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.681922    2571 factory.go:153] Registering CRI-O factory
May 25 13:59:52.682059 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.682019    2571 factory.go:223] Registration of the crio container factory successfully
May 25 13:59:52.682124 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.682082    2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 25 13:59:52.682124 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.682105    2571 factory.go:103] Registering Raw factory
May 25 13:59:52.682124 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.682121    2571 manager.go:1196] Started watching for new ooms in manager
May 25 13:59:52.682124 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:52.681982    2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-220.ec2.internal\" not found"
May 25 13:59:52.682571 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.682556    2571 manager.go:319] Starting recovery of all containers
May 25 13:59:52.683629 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:52.683605    2571 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
May 25 13:59:52.683717 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:52.683628    2571 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-220.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
May 25 13:59:52.689599 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.689569    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pp2lq"
May 25 13:59:52.690622 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.690606    2571 manager.go:324] Recovery completed
May 25 13:59:52.696083 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.696065    2571 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 25 13:59:52.697971 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.697950    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pp2lq"
May 25 13:59:52.698114 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.698101    2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-220.ec2.internal" event="NodeHasSufficientMemory"
May 25 13:59:52.698179 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.698135    2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-220.ec2.internal" event="NodeHasNoDiskPressure"
May 25 13:59:52.698179 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.698151    2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-220.ec2.internal" event="NodeHasSufficientPID"
May 25 13:59:52.698767 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.698738    2571 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 25 13:59:52.698767 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.698765    2571 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 25 13:59:52.698857 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.698781    2571 state_mem.go:36] "Initialized new in-memory state store"
May 25 13:59:52.702889 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.702875    2571 policy_none.go:49] "None policy: Start"
May 25 13:59:52.702942 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.702894    2571 memory_manager.go:186] "Starting memorymanager" policy="None"
May 25 13:59:52.702942 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.702906    2571 state_mem.go:35] "Initializing new in-memory state store"
May 25 13:59:52.746696 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.746674    2571 manager.go:341] "Starting Device Plugin manager"
May 25 13:59:52.773831 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:52.746772    2571 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 25 13:59:52.773831 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.746786    2571 server.go:85] "Starting device plugin registration server"
May 25 13:59:52.773831 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.747051    2571 eviction_manager.go:189] "Eviction manager: starting control loop"
May 25 13:59:52.773831 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.747063    2571 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 25 13:59:52.773831 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.747166    2571 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 25 13:59:52.773831 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.747282    2571 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 25 13:59:52.773831 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.747294    2571 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 25 13:59:52.773831 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:52.748222    2571 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 25 13:59:52.773831 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:52.748264    2571 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-220.ec2.internal\" not found"
May 25 13:59:52.819740 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.819648    2571 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 25 13:59:52.821093 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.821071    2571 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 25 13:59:52.821236 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.821120    2571 status_manager.go:230] "Starting to sync pod status with apiserver"
May 25 13:59:52.821236 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.821147    2571 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 25 13:59:52.821236 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.821159    2571 kubelet.go:2451] "Starting kubelet main sync loop"
May 25 13:59:52.821367 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:52.821272    2571 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 25 13:59:52.827520 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.827498    2571 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 25 13:59:52.848004 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.847964    2571 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 25 13:59:52.848961 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.848942    2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-220.ec2.internal" event="NodeHasSufficientMemory"
May 25 13:59:52.849057 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.848976    2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-220.ec2.internal" event="NodeHasNoDiskPressure"
May 25 13:59:52.849057 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.848988    2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-220.ec2.internal" event="NodeHasSufficientPID"
May 25 13:59:52.849057 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.849013    2571 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-220.ec2.internal"
May 25 13:59:52.856135 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.856115    2571 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-220.ec2.internal"
May 25 13:59:52.856256 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:52.856180    2571 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-220.ec2.internal\": node \"ip-10-0-128-220.ec2.internal\" not found"
May 25 13:59:52.878866 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:52.878833    2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-220.ec2.internal\" not found"
May 25 13:59:52.922023 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.921896    2571 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-220.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-220.ec2.internal"]
May 25 13:59:52.922319 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.922122    2571 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 25 13:59:52.931013 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.930966    2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-220.ec2.internal" event="NodeHasSufficientMemory"
May 25 13:59:52.931013 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.931029    2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-220.ec2.internal" event="NodeHasNoDiskPressure"
May 25 13:59:52.931445 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.931043    2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-220.ec2.internal" event="NodeHasSufficientPID"
May 25 13:59:52.932788 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.932750    2571 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 25 13:59:52.933140 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.933106    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-220.ec2.internal"
May 25 13:59:52.933275 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.933199    2571 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 25 13:59:52.934497 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.934457    2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-220.ec2.internal" event="NodeHasSufficientMemory"
May 25 13:59:52.934705 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.934523    2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-220.ec2.internal" event="NodeHasNoDiskPressure"
May 25 13:59:52.934705 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.934539    2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-220.ec2.internal" event="NodeHasSufficientPID"
May 25 13:59:52.934705 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.934457    2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-220.ec2.internal" event="NodeHasSufficientMemory"
May 25 13:59:52.934705 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.934587    2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-220.ec2.internal" event="NodeHasNoDiskPressure"
May 25 13:59:52.934705 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.934601    2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-220.ec2.internal" event="NodeHasSufficientPID"
May 25 13:59:52.936333 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.936300    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-220.ec2.internal"
May 25 13:59:52.936463 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.936366    2571 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 25 13:59:52.938145 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.938105    2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-220.ec2.internal" event="NodeHasSufficientMemory"
May 25 13:59:52.938287 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.938181    2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-220.ec2.internal" event="NodeHasNoDiskPressure"
May 25 13:59:52.938287 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.938200    2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-220.ec2.internal" event="NodeHasSufficientPID"
May 25 13:59:52.956706 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:52.956676    2571 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-220.ec2.internal\" not found" node="ip-10-0-128-220.ec2.internal"
May 25 13:59:52.960529 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:52.960510    2571 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-220.ec2.internal\" not found" node="ip-10-0-128-220.ec2.internal"
May 25 13:59:52.979559 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:52.979528    2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-220.ec2.internal\" not found"
May 25 13:59:52.982841 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.982822    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/50b522fb9d3992d5ac1c179e7492fa8c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-220.ec2.internal\" (UID: \"50b522fb9d3992d5ac1c179e7492fa8c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-220.ec2.internal"
May 25 13:59:52.982905 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.982851    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50b522fb9d3992d5ac1c179e7492fa8c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-220.ec2.internal\" (UID: \"50b522fb9d3992d5ac1c179e7492fa8c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-220.ec2.internal"
May 25 13:59:52.982905 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:52.982871    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9707757169265407340417083d330c50-config\") pod \"kube-apiserver-proxy-ip-10-0-128-220.ec2.internal\" (UID: \"9707757169265407340417083d330c50\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-220.ec2.internal"
May 25 13:59:53.080473 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:53.080385    2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-220.ec2.internal\" not found"
May 25 13:59:53.083710 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:53.083692    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50b522fb9d3992d5ac1c179e7492fa8c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-220.ec2.internal\" (UID: \"50b522fb9d3992d5ac1c179e7492fa8c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-220.ec2.internal"
May 25 13:59:53.083763 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:53.083719    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9707757169265407340417083d330c50-config\") pod \"kube-apiserver-proxy-ip-10-0-128-220.ec2.internal\" (UID: \"9707757169265407340417083d330c50\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-220.ec2.internal"
May 25 13:59:53.083763 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:53.083746    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/50b522fb9d3992d5ac1c179e7492fa8c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-220.ec2.internal\" (UID: \"50b522fb9d3992d5ac1c179e7492fa8c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-220.ec2.internal"
May 25 13:59:53.083827 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:53.083796    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9707757169265407340417083d330c50-config\") pod \"kube-apiserver-proxy-ip-10-0-128-220.ec2.internal\" (UID: \"9707757169265407340417083d330c50\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-220.ec2.internal"
May 25 13:59:53.083827 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:53.083805    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50b522fb9d3992d5ac1c179e7492fa8c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-220.ec2.internal\" (UID: \"50b522fb9d3992d5ac1c179e7492fa8c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-220.ec2.internal"
May 25 13:59:53.083886 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:53.083844    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/50b522fb9d3992d5ac1c179e7492fa8c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-220.ec2.internal\" (UID: \"50b522fb9d3992d5ac1c179e7492fa8c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-220.ec2.internal"
May 25 13:59:53.181210 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:53.181168    2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-220.ec2.internal\" not found"
May 25 13:59:53.259611 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:53.259565    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-220.ec2.internal"
May 25 13:59:53.263446 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:53.263419    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-220.ec2.internal"
May 25 13:59:53.281279 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:53.281237    2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-220.ec2.internal\" not found"
May 25 13:59:53.381729 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:53.381622    2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-220.ec2.internal\" not found"
May 25 13:59:53.482103 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:53.482070    2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-220.ec2.internal\" not found"
May 25 13:59:53.575465 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:53.575410    2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 25 13:59:53.576186 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:53.575595    2571 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 25 13:59:53.582691 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:53.582661    2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-220.ec2.internal\" not found"
May 25 13:59:53.681275 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:53.681176    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 25 13:59:53.683361 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:53.683340    2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-220.ec2.internal\" not found"
May 25 13:59:53.689093 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:53.689073    2571 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 25 13:59:53.700636 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:53.700583    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-24 13:54:52 +0000 UTC" deadline="2027-11-21 01:39:53.843814063 +0000 UTC"
May 25 13:59:53.700636 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:53.700625    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13067h40m0.143192466s"
May 25 13:59:53.713256 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:53.713224    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-s5bx6"
May 25 13:59:53.721037 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:53.721006    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-s5bx6"
May 25 13:59:53.784394 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:53.784215    2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-220.ec2.internal\" not found"
May 25 13:59:53.866620 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:53.866597    2571 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 25 13:59:53.878006 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:53.877959    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9707757169265407340417083d330c50.slice/crio-810bc04c226a48aaa0472360e5f869c74e1d9d671d89347ff1a5173a0ce91a97 WatchSource:0}: Error finding container 810bc04c226a48aaa0472360e5f869c74e1d9d671d89347ff1a5173a0ce91a97: Status 404 returned error can't find the container with id 810bc04c226a48aaa0472360e5f869c74e1d9d671d89347ff1a5173a0ce91a97
May 25 13:59:53.878271 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:53.878248    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50b522fb9d3992d5ac1c179e7492fa8c.slice/crio-1ba69bd9d3ab0e5d8f4f4deece3d432ee80e7e21aa42d63c4abe82078f18ef2d WatchSource:0}: Error finding container 1ba69bd9d3ab0e5d8f4f4deece3d432ee80e7e21aa42d63c4abe82078f18ef2d: Status 404 returned error can't find the container with id 1ba69bd9d3ab0e5d8f4f4deece3d432ee80e7e21aa42d63c4abe82078f18ef2d
May 25 13:59:53.882626 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:53.882608    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 25 13:59:53.885250 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:53.885232    2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-220.ec2.internal\" not found"
May 25 13:59:53.986298 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:53.986252    2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-220.ec2.internal\" not found"
May 25 13:59:53.997668 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:53.996456    2571 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 25 13:59:54.082185 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.082148    2571 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-220.ec2.internal"
May 25 13:59:54.092057 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.092031    2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 25 13:59:54.094096 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.094082    2571 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-220.ec2.internal"
May 25 13:59:54.107309 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.107280    2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 25 13:59:54.112298 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.112273    2571 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 25 13:59:54.654037 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.654004    2571 apiserver.go:52] "Watching apiserver"
May 25 13:59:54.659240 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.659214    2571 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 25 13:59:54.661857 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.661823    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-lc2jb","kube-system/kube-apiserver-proxy-ip-10-0-128-220.ec2.internal","openshift-cluster-node-tuning-operator/tuned-bw5st","openshift-dns/node-resolver-rx92j","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-220.ec2.internal","openshift-multus/multus-zm52l","openshift-network-diagnostics/network-check-target-fb2gw","openshift-network-operator/iptables-alerter-bt7c9","openshift-ovn-kubernetes/ovnkube-node-fstv5","kube-system/konnectivity-agent-c8rbn","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm","openshift-image-registry/node-ca-7jmvx","openshift-multus/multus-additional-cni-plugins-msrg9"]
May 25 13:59:54.666551 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.666526    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 13:59:54.666683 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:54.666614    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2gw" podUID="98f00859-da06-410b-8572-c64348ab32d5"
May 25 13:59:54.668598 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.668572    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rx92j"
May 25 13:59:54.671734 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.670550    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 25 13:59:54.671734 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.670572    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 25 13:59:54.671734 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.670821    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mtngh\""
May 25 13:59:54.671734 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.670873    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.672546 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.672528    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 25 13:59:54.672738 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.672717    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zkkfc\""
May 25 13:59:54.672866 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.672851    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 25 13:59:54.673116 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.673099    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 25 13:59:54.673168 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.673116    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 25 13:59:54.673363 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.673341    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 25 13:59:54.673363 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.673359    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c8rbn"
May 25 13:59:54.673520 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.673462    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 25 13:59:54.675485 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.675459    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 25 13:59:54.676785 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.676763    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.677782 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.677767    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 25 13:59:54.679528 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.679503    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zm52l"
May 25 13:59:54.679655 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.679635    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 25 13:59:54.679950 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.679917    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 25 13:59:54.680565 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.680144    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-v2brd\""
May 25 13:59:54.680565 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.680152    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xdwrm\""
May 25 13:59:54.681299 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.681270    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 25 13:59:54.681502 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.681478    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-687t7\""
May 25 13:59:54.681590 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.681549    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 25 13:59:54.681961 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.681926    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 25 13:59:54.682321 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.682302    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 25 13:59:54.683844 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.683826    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 13:59:54.683950 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:54.683914    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc2jb" podUID="f9f60fa8-15a3-402e-842e-f1a52b7a3404"
May 25 13:59:54.684015 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.683965    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bt7c9"
May 25 13:59:54.685493 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.685474    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 25 13:59:54.685689 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.685672    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 25 13:59:54.685759 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.685737    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bngzt\""
May 25 13:59:54.685949 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.685919    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 25 13:59:54.688546 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.688521    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm"
May 25 13:59:54.688634 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.688599    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7jmvx"
May 25 13:59:54.690420 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.690389    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 25 13:59:54.690569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.690555    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 25 13:59:54.690742 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.690728    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7ttb4\""
May 25 13:59:54.690793 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.690764    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 25 13:59:54.691049 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.691009    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 25 13:59:54.691186 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.691171    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 25 13:59:54.691256 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.691211    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 25 13:59:54.691315 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.691299    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-nnwwl\""
May 25 13:59:54.691735 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.691697    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-msrg9"
May 25 13:59:54.693379 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.693360    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 25 13:59:54.693466 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.693431    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 25 13:59:54.693595 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.693576    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fnqsr\""
May 25 13:59:54.693694 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.693599    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-host\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.693694 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.693630    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-host-var-lib-cni-multus\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.693694 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.693659    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-host-var-lib-kubelet\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.693694 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.693684    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-multus-conf-dir\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.693889 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.693719    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-etc-systemd\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.693889 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.693745    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-host-kubelet\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.693889 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.693785    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-run-openvswitch\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.693889 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.693811    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrfj4\" (UniqueName: \"kubernetes.io/projected/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-kube-api-access-hrfj4\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.693889 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.693855    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-etc-kubernetes\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.693889 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.693873    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-etc-sysctl-conf\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.693889 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.693887    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-var-lib-kubelet\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.694250 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.693910    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dxnb\" (UniqueName: \"kubernetes.io/projected/f9f60fa8-15a3-402e-842e-f1a52b7a3404-kube-api-access-8dxnb\") pod \"network-metrics-daemon-lc2jb\" (UID: \"f9f60fa8-15a3-402e-842e-f1a52b7a3404\") " pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 13:59:54.694250 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.693927    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-multus-socket-dir-parent\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.694250 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.693983    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9f60fa8-15a3-402e-842e-f1a52b7a3404-metrics-certs\") pod \"network-metrics-daemon-lc2jb\" (UID: \"f9f60fa8-15a3-402e-842e-f1a52b7a3404\") " pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 13:59:54.694250 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694005    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-var-lib-openvswitch\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.694250 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694031    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-node-log\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.694250 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694056    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-ovnkube-config\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.694250 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694076    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-cni-binary-copy\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.694250 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694092    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68251ce2-60d3-4c72-8e3f-58d0ea9d9629-tmp-dir\") pod \"node-resolver-rx92j\" (UID: \"68251ce2-60d3-4c72-8e3f-58d0ea9d9629\") " pod="openshift-dns/node-resolver-rx92j"
May 25 13:59:54.694250 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694109    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-etc-kubernetes\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.694250 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694132    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/68251ce2-60d3-4c72-8e3f-58d0ea9d9629-hosts-file\") pod \"node-resolver-rx92j\" (UID: \"68251ce2-60d3-4c72-8e3f-58d0ea9d9629\") " pod="openshift-dns/node-resolver-rx92j"
May 25 13:59:54.694250 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694155    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-etc-sysconfig\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.694250 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694178    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmr7j\" (UniqueName: \"kubernetes.io/projected/198a8047-e88d-4bc2-805d-0bfd6155fb0f-kube-api-access-jmr7j\") pod \"iptables-alerter-bt7c9\" (UID: \"198a8047-e88d-4bc2-805d-0bfd6155fb0f\") " pod="openshift-network-operator/iptables-alerter-bt7c9"
May 25 13:59:54.694250 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694245    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-system-cni-dir\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.694923 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694279    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-host-run-multus-certs\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.694923 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694309    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-systemd-units\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.694923 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694333    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-host-run-netns\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.694923 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694359    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-etc-openvswitch\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.694923 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694382    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-host-cni-bin\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.694923 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694408    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kvdt\" (UniqueName: \"kubernetes.io/projected/68251ce2-60d3-4c72-8e3f-58d0ea9d9629-kube-api-access-2kvdt\") pod \"node-resolver-rx92j\" (UID: \"68251ce2-60d3-4c72-8e3f-58d0ea9d9629\") " pod="openshift-dns/node-resolver-rx92j"
May 25 13:59:54.694923 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694431    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-etc-sysctl-d\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.694923 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694453    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-lib-modules\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.694923 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694476    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-cnibin\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.694923 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694510    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.694923 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694537    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-host-run-k8s-cni-cncf-io\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.694923 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694561    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qb8wp\" (UniqueName: \"kubernetes.io/projected/e3d0b072-8bba-48c2-98cc-d14341527f7c-kube-api-access-qb8wp\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.694923 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694585    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-env-overrides\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.694923 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694613    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-etc-modprobe-d\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.694923 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694650    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs7zp\" (UniqueName: \"kubernetes.io/projected/98f00859-da06-410b-8572-c64348ab32d5-kube-api-access-gs7zp\") pod \"network-check-target-fb2gw\" (UID: \"98f00859-da06-410b-8572-c64348ab32d5\") " pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 13:59:54.694923 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694688    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-run-systemd\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.695661 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694718    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-sys\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.695661 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694793    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-log-socket\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.695661 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694807    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-ovn-node-metrics-cert\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.695661 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694821    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-ovnkube-script-lib\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.695661 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694836    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-os-release\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.695661 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694853    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-host-run-netns\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.695661 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694866    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-run\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.695661 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694879    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-host-cni-netd\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.695661 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694893    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-hostroot\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.695661 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694906    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e3d0b072-8bba-48c2-98cc-d14341527f7c-tmp\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.695661 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694920    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/198a8047-e88d-4bc2-805d-0bfd6155fb0f-iptables-alerter-script\") pod \"iptables-alerter-bt7c9\" (UID: \"198a8047-e88d-4bc2-805d-0bfd6155fb0f\") " pod="openshift-network-operator/iptables-alerter-bt7c9"
May 25 13:59:54.695661 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694954    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a0a2025d-c91e-4fd7-a82a-37a9416c9fc5-konnectivity-ca\") pod \"konnectivity-agent-c8rbn\" (UID: \"a0a2025d-c91e-4fd7-a82a-37a9416c9fc5\") " pod="kube-system/konnectivity-agent-c8rbn"
May 25 13:59:54.695661 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.694983    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e3d0b072-8bba-48c2-98cc-d14341527f7c-etc-tuned\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.695661 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.695008    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/198a8047-e88d-4bc2-805d-0bfd6155fb0f-host-slash\") pod \"iptables-alerter-bt7c9\" (UID: \"198a8047-e88d-4bc2-805d-0bfd6155fb0f\") " pod="openshift-network-operator/iptables-alerter-bt7c9"
May 25 13:59:54.695661 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.695031    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-host-slash\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.695661 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.695047    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a0a2025d-c91e-4fd7-a82a-37a9416c9fc5-agent-certs\") pod \"konnectivity-agent-c8rbn\" (UID: \"a0a2025d-c91e-4fd7-a82a-37a9416c9fc5\") " pod="kube-system/konnectivity-agent-c8rbn"
May 25 13:59:54.695661 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.695064    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-host-var-lib-cni-bin\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.696336 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.695083    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-multus-daemon-config\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.696336 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.695105    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf46m\" (UniqueName: \"kubernetes.io/projected/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-kube-api-access-bf46m\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.696336 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.695128    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-multus-cni-dir\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.696336 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.695153    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-run-ovn\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.696336 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.695217    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.709638 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.709609    2571 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 25 13:59:54.721788 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.721753    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-24 13:54:53 +0000 UTC" deadline="2028-02-11 11:25:12.579926844 +0000 UTC"
May 25 13:59:54.721788 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.721785    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15045h25m17.858144776s"
May 25 13:59:54.782229 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.782192    2571 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 25 13:59:54.795786 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.795751    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-var-lib-openvswitch\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.795983 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.795810    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-node-log\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.795983 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.795837    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-var-lib-openvswitch\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.795983 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.795840    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-ovnkube-config\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.795983 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.795905    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-cni-binary-copy\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.795983 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.795920    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-node-log\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.795983 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.795945    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68251ce2-60d3-4c72-8e3f-58d0ea9d9629-tmp-dir\") pod \"node-resolver-rx92j\" (UID: \"68251ce2-60d3-4c72-8e3f-58d0ea9d9629\") " pod="openshift-dns/node-resolver-rx92j"
May 25 13:59:54.795983 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.795983    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b8d826df-35ec-4d3f-9b5b-eea218065da9-registration-dir\") pod \"aws-ebs-csi-driver-node-926nm\" (UID: \"b8d826df-35ec-4d3f-9b5b-eea218065da9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm"
May 25 13:59:54.796348 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796012    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-etc-kubernetes\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.796348 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796040    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/68251ce2-60d3-4c72-8e3f-58d0ea9d9629-hosts-file\") pod \"node-resolver-rx92j\" (UID: \"68251ce2-60d3-4c72-8e3f-58d0ea9d9629\") " pod="openshift-dns/node-resolver-rx92j"
May 25 13:59:54.796348 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796062    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-etc-sysconfig\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.796348 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796085    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jmr7j\" (UniqueName: \"kubernetes.io/projected/198a8047-e88d-4bc2-805d-0bfd6155fb0f-kube-api-access-jmr7j\") pod \"iptables-alerter-bt7c9\" (UID: \"198a8047-e88d-4bc2-805d-0bfd6155fb0f\") " pod="openshift-network-operator/iptables-alerter-bt7c9"
May 25 13:59:54.796348 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796110    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b8d826df-35ec-4d3f-9b5b-eea218065da9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-926nm\" (UID: \"b8d826df-35ec-4d3f-9b5b-eea218065da9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm"
May 25 13:59:54.796348 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796134    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/370bcb80-f526-48ae-8582-1d104d764c96-host\") pod \"node-ca-7jmvx\" (UID: \"370bcb80-f526-48ae-8582-1d104d764c96\") " pod="openshift-image-registry/node-ca-7jmvx"
May 25 13:59:54.796348 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796160    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-system-cni-dir\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.796348 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796193    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-host-run-multus-certs\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.796348 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796218    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f-system-cni-dir\") pod \"multus-additional-cni-plugins-msrg9\" (UID: \"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f\") " pod="openshift-multus/multus-additional-cni-plugins-msrg9"
May 25 13:59:54.796348 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796242    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f-os-release\") pod \"multus-additional-cni-plugins-msrg9\" (UID: \"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f\") " pod="openshift-multus/multus-additional-cni-plugins-msrg9"
May 25 13:59:54.796348 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796262    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68251ce2-60d3-4c72-8e3f-58d0ea9d9629-tmp-dir\") pod \"node-resolver-rx92j\" (UID: \"68251ce2-60d3-4c72-8e3f-58d0ea9d9629\") " pod="openshift-dns/node-resolver-rx92j"
May 25 13:59:54.796348 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796270    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-systemd-units\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.796348 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796296    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-host-run-netns\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.796348 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796336    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-etc-openvswitch\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.797006 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796371    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-host-cni-bin\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.797006 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796395    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2kvdt\" (UniqueName: \"kubernetes.io/projected/68251ce2-60d3-4c72-8e3f-58d0ea9d9629-kube-api-access-2kvdt\") pod \"node-resolver-rx92j\" (UID: \"68251ce2-60d3-4c72-8e3f-58d0ea9d9629\") " pod="openshift-dns/node-resolver-rx92j"
May 25 13:59:54.797006 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796420    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-etc-sysctl-d\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.797006 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796444    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-lib-modules\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.797006 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796467    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-cnibin\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.797006 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796491    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.797006 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796518    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-host-run-k8s-cni-cncf-io\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.797006 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796545    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qb8wp\" (UniqueName: \"kubernetes.io/projected/e3d0b072-8bba-48c2-98cc-d14341527f7c-kube-api-access-qb8wp\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.797006 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796576    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-msrg9\" (UID: \"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f\") " pod="openshift-multus/multus-additional-cni-plugins-msrg9"
May 25 13:59:54.797006 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796600    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b8d826df-35ec-4d3f-9b5b-eea218065da9-sys-fs\") pod \"aws-ebs-csi-driver-node-926nm\" (UID: \"b8d826df-35ec-4d3f-9b5b-eea218065da9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm"
May 25 13:59:54.797006 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796628    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-env-overrides\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.797006 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796654    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-etc-modprobe-d\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.797006 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796685    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gs7zp\" (UniqueName: \"kubernetes.io/projected/98f00859-da06-410b-8572-c64348ab32d5-kube-api-access-gs7zp\") pod \"network-check-target-fb2gw\" (UID: \"98f00859-da06-410b-8572-c64348ab32d5\") " pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 13:59:54.797006 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796691    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-ovnkube-config\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.797006 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796707    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-cni-binary-copy\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.797006 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796710    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b8d826df-35ec-4d3f-9b5b-eea218065da9-device-dir\") pod \"aws-ebs-csi-driver-node-926nm\" (UID: \"b8d826df-35ec-4d3f-9b5b-eea218065da9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm"
May 25 13:59:54.797006 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796745    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-run-systemd\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.797741 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796755    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-etc-kubernetes\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.797741 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796796    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-msrg9\" (UID: \"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f\") " pod="openshift-multus/multus-additional-cni-plugins-msrg9"
May 25 13:59:54.797741 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796820    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5l7z\" (UniqueName: \"kubernetes.io/projected/370bcb80-f526-48ae-8582-1d104d764c96-kube-api-access-f5l7z\") pod \"node-ca-7jmvx\" (UID: \"370bcb80-f526-48ae-8582-1d104d764c96\") " pod="openshift-image-registry/node-ca-7jmvx"
May 25 13:59:54.797741 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796825    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-run-systemd\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.797741 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796832    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-cnibin\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.797741 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796861    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-host-cni-bin\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.797741 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796870    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-etc-openvswitch\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.797741 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796878    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-sys\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.797741 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796897    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-host-run-multus-certs\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.797741 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796901    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-etc-sysconfig\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.797741 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796913    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-host-run-netns\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.797741 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796955    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-systemd-units\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.797741 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796839    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-sys\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.797741 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.796991    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/68251ce2-60d3-4c72-8e3f-58d0ea9d9629-hosts-file\") pod \"node-resolver-rx92j\" (UID: \"68251ce2-60d3-4c72-8e3f-58d0ea9d9629\") " pod="openshift-dns/node-resolver-rx92j"
May 25 13:59:54.797741 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.797015    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-system-cni-dir\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.797741 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.797196    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-env-overrides\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.797741 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.797268    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-etc-sysctl-d\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.797741 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.797269    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-etc-modprobe-d\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.798507 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.797351    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-lib-modules\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.798507 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.797382    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.798507 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.797397    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-host-run-k8s-cni-cncf-io\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.798507 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.797434    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-log-socket\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.798507 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.797473    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-ovn-node-metrics-cert\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.798507 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.797530    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-ovnkube-script-lib\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.798507 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.797539    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-log-socket\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.798507 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.797569    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-os-release\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.798507 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.797675    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-host-run-netns\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.798507 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.797732    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-run\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.798507 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.797732    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-os-release\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.798507 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.797759    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-host-run-netns\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.798507 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.797798    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-host-cni-netd\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.798507 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.797846    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-hostroot\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.798507 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.797831    2571 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 25 13:59:54.798507 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.797907    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-host-cni-netd\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.798507 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.797906    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e3d0b072-8bba-48c2-98cc-d14341527f7c-tmp\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.798507 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.797927    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-hostroot\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.799365 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.797986    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/198a8047-e88d-4bc2-805d-0bfd6155fb0f-iptables-alerter-script\") pod \"iptables-alerter-bt7c9\" (UID: \"198a8047-e88d-4bc2-805d-0bfd6155fb0f\") " pod="openshift-network-operator/iptables-alerter-bt7c9"
May 25 13:59:54.799365 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798024    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f-cni-binary-copy\") pod \"multus-additional-cni-plugins-msrg9\" (UID: \"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f\") " pod="openshift-multus/multus-additional-cni-plugins-msrg9"
May 25 13:59:54.799365 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798034    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-ovnkube-script-lib\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.799365 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.797862    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-run\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.799365 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798059    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-msrg9\" (UID: \"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f\") " pod="openshift-multus/multus-additional-cni-plugins-msrg9"
May 25 13:59:54.799365 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798160    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a0a2025d-c91e-4fd7-a82a-37a9416c9fc5-konnectivity-ca\") pod \"konnectivity-agent-c8rbn\" (UID: \"a0a2025d-c91e-4fd7-a82a-37a9416c9fc5\") " pod="kube-system/konnectivity-agent-c8rbn"
May 25 13:59:54.799365 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798187    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e3d0b072-8bba-48c2-98cc-d14341527f7c-etc-tuned\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.799365 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798240    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/198a8047-e88d-4bc2-805d-0bfd6155fb0f-host-slash\") pod \"iptables-alerter-bt7c9\" (UID: \"198a8047-e88d-4bc2-805d-0bfd6155fb0f\") " pod="openshift-network-operator/iptables-alerter-bt7c9"
May 25 13:59:54.799365 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798269    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tn656\" (UniqueName: \"kubernetes.io/projected/dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f-kube-api-access-tn656\") pod \"multus-additional-cni-plugins-msrg9\" (UID: \"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f\") " pod="openshift-multus/multus-additional-cni-plugins-msrg9"
May 25 13:59:54.799365 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798314    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b8d826df-35ec-4d3f-9b5b-eea218065da9-etc-selinux\") pod \"aws-ebs-csi-driver-node-926nm\" (UID: \"b8d826df-35ec-4d3f-9b5b-eea218065da9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm"
May 25 13:59:54.799365 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798366    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-host-slash\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.799365 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798389    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a0a2025d-c91e-4fd7-a82a-37a9416c9fc5-agent-certs\") pod \"konnectivity-agent-c8rbn\" (UID: \"a0a2025d-c91e-4fd7-a82a-37a9416c9fc5\") " pod="kube-system/konnectivity-agent-c8rbn"
May 25 13:59:54.799365 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798413    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-host-var-lib-cni-bin\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.799365 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798434    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/198a8047-e88d-4bc2-805d-0bfd6155fb0f-host-slash\") pod \"iptables-alerter-bt7c9\" (UID: \"198a8047-e88d-4bc2-805d-0bfd6155fb0f\") " pod="openshift-network-operator/iptables-alerter-bt7c9"
May 25 13:59:54.799365 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798438    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-multus-daemon-config\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.799365 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798475    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87mpp\" (UniqueName: \"kubernetes.io/projected/b8d826df-35ec-4d3f-9b5b-eea218065da9-kube-api-access-87mpp\") pod \"aws-ebs-csi-driver-node-926nm\" (UID: \"b8d826df-35ec-4d3f-9b5b-eea218065da9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm"
May 25 13:59:54.799365 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798501    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bf46m\" (UniqueName: \"kubernetes.io/projected/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-kube-api-access-bf46m\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.800156 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798526    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-multus-cni-dir\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.800156 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798534    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/198a8047-e88d-4bc2-805d-0bfd6155fb0f-iptables-alerter-script\") pod \"iptables-alerter-bt7c9\" (UID: \"198a8047-e88d-4bc2-805d-0bfd6155fb0f\") " pod="openshift-network-operator/iptables-alerter-bt7c9"
May 25 13:59:54.800156 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798550    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-run-ovn\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.800156 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798578    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.800156 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798595    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-host-slash\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.800156 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798600    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-host\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.800156 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798635    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f-cnibin\") pod \"multus-additional-cni-plugins-msrg9\" (UID: \"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f\") " pod="openshift-multus/multus-additional-cni-plugins-msrg9"
May 25 13:59:54.800156 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798651    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-multus-cni-dir\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.800156 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798655    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-host-var-lib-cni-multus\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.800156 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798680    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-host-var-lib-cni-multus\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.800156 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798692    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-host-var-lib-kubelet\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.800156 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798717    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-multus-conf-dir\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.800156 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798743    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-etc-systemd\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.800156 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798755    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a0a2025d-c91e-4fd7-a82a-37a9416c9fc5-konnectivity-ca\") pod \"konnectivity-agent-c8rbn\" (UID: \"a0a2025d-c91e-4fd7-a82a-37a9416c9fc5\") " pod="kube-system/konnectivity-agent-c8rbn"
May 25 13:59:54.800156 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798770    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b8d826df-35ec-4d3f-9b5b-eea218065da9-socket-dir\") pod \"aws-ebs-csi-driver-node-926nm\" (UID: \"b8d826df-35ec-4d3f-9b5b-eea218065da9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm"
May 25 13:59:54.800156 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798807    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-host-kubelet\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.800156 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798832    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-run-ovn\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.800156 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798831    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-host\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.801035 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798837    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-run-openvswitch\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.801035 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798875    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.801035 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798878    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-multus-daemon-config\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.801035 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798886    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-host-var-lib-kubelet\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.801035 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798897    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-host-kubelet\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.801035 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798919    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-multus-conf-dir\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.801035 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798923    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hrfj4\" (UniqueName: \"kubernetes.io/projected/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-kube-api-access-hrfj4\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.801035 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798927    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-host-var-lib-cni-bin\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.801035 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798980    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-run-openvswitch\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.801035 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.798998    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-etc-systemd\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.801035 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.799026    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-etc-kubernetes\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.801035 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.799049    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-etc-sysctl-conf\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.801035 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.799064    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-var-lib-kubelet\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.801035 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.799076    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-etc-kubernetes\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.801035 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.799086    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8dxnb\" (UniqueName: \"kubernetes.io/projected/f9f60fa8-15a3-402e-842e-f1a52b7a3404-kube-api-access-8dxnb\") pod \"network-metrics-daemon-lc2jb\" (UID: \"f9f60fa8-15a3-402e-842e-f1a52b7a3404\") " pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 13:59:54.801035 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.799113    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/370bcb80-f526-48ae-8582-1d104d764c96-serviceca\") pod \"node-ca-7jmvx\" (UID: \"370bcb80-f526-48ae-8582-1d104d764c96\") " pod="openshift-image-registry/node-ca-7jmvx"
May 25 13:59:54.801035 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.799122    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-var-lib-kubelet\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.801035 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.799135    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-multus-socket-dir-parent\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.801831 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.799165    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9f60fa8-15a3-402e-842e-f1a52b7a3404-metrics-certs\") pod \"network-metrics-daemon-lc2jb\" (UID: \"f9f60fa8-15a3-402e-842e-f1a52b7a3404\") " pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 13:59:54.801831 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.799187    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e3d0b072-8bba-48c2-98cc-d14341527f7c-etc-sysctl-conf\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.801831 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:54.799237    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 13:59:54.801831 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.799247    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-multus-socket-dir-parent\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.801831 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:54.799364    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9f60fa8-15a3-402e-842e-f1a52b7a3404-metrics-certs podName:f9f60fa8-15a3-402e-842e-f1a52b7a3404 nodeName:}" failed. No retries permitted until 2026-05-25 13:59:55.299349233 +0000 UTC m=+3.134205320 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f9f60fa8-15a3-402e-842e-f1a52b7a3404-metrics-certs") pod "network-metrics-daemon-lc2jb" (UID: "f9f60fa8-15a3-402e-842e-f1a52b7a3404") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 13:59:54.802302 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.802277    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a0a2025d-c91e-4fd7-a82a-37a9416c9fc5-agent-certs\") pod \"konnectivity-agent-c8rbn\" (UID: \"a0a2025d-c91e-4fd7-a82a-37a9416c9fc5\") " pod="kube-system/konnectivity-agent-c8rbn"
May 25 13:59:54.802554 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:54.802531    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 13:59:54.802554 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:54.802556    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 13:59:54.802795 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:54.802567    2571 projected.go:194] Error preparing data for projected volume kube-api-access-gs7zp for pod openshift-network-diagnostics/network-check-target-fb2gw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 13:59:54.802795 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:54.802634    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98f00859-da06-410b-8572-c64348ab32d5-kube-api-access-gs7zp podName:98f00859-da06-410b-8572-c64348ab32d5 nodeName:}" failed. No retries permitted until 2026-05-25 13:59:55.302621195 +0000 UTC m=+3.137477270 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gs7zp" (UniqueName: "kubernetes.io/projected/98f00859-da06-410b-8572-c64348ab32d5-kube-api-access-gs7zp") pod "network-check-target-fb2gw" (UID: "98f00859-da06-410b-8572-c64348ab32d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 13:59:54.806424 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.806398    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qb8wp\" (UniqueName: \"kubernetes.io/projected/e3d0b072-8bba-48c2-98cc-d14341527f7c-kube-api-access-qb8wp\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.807064 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.806839    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-ovn-node-metrics-cert\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.807733 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.807581    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kvdt\" (UniqueName: \"kubernetes.io/projected/68251ce2-60d3-4c72-8e3f-58d0ea9d9629-kube-api-access-2kvdt\") pod \"node-resolver-rx92j\" (UID: \"68251ce2-60d3-4c72-8e3f-58d0ea9d9629\") " pod="openshift-dns/node-resolver-rx92j"
May 25 13:59:54.808029 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.808003    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dxnb\" (UniqueName: \"kubernetes.io/projected/f9f60fa8-15a3-402e-842e-f1a52b7a3404-kube-api-access-8dxnb\") pod \"network-metrics-daemon-lc2jb\" (UID: \"f9f60fa8-15a3-402e-842e-f1a52b7a3404\") " pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 13:59:54.808438 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.808416    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e3d0b072-8bba-48c2-98cc-d14341527f7c-tmp\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.809686 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.809664    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf46m\" (UniqueName: \"kubernetes.io/projected/c7a16d3f-06d4-4855-bd37-0e1df89d84b5-kube-api-access-bf46m\") pod \"ovnkube-node-fstv5\" (UID: \"c7a16d3f-06d4-4855-bd37-0e1df89d84b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:54.809959 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.809925    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e3d0b072-8bba-48c2-98cc-d14341527f7c-etc-tuned\") pod \"tuned-bw5st\" (UID: \"e3d0b072-8bba-48c2-98cc-d14341527f7c\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:54.810612 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.810588    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmr7j\" (UniqueName: \"kubernetes.io/projected/198a8047-e88d-4bc2-805d-0bfd6155fb0f-kube-api-access-jmr7j\") pod \"iptables-alerter-bt7c9\" (UID: \"198a8047-e88d-4bc2-805d-0bfd6155fb0f\") " pod="openshift-network-operator/iptables-alerter-bt7c9"
May 25 13:59:54.810681 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.810599    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrfj4\" (UniqueName: \"kubernetes.io/projected/6e393b7c-9ac9-4848-95f7-7aa7845d67d9-kube-api-access-hrfj4\") pod \"multus-zm52l\" (UID: \"6e393b7c-9ac9-4848-95f7-7aa7845d67d9\") " pod="openshift-multus/multus-zm52l"
May 25 13:59:54.825466 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.825388    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-220.ec2.internal" event={"ID":"9707757169265407340417083d330c50","Type":"ContainerStarted","Data":"810bc04c226a48aaa0472360e5f869c74e1d9d671d89347ff1a5173a0ce91a97"}
May 25 13:59:54.826399 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.826360    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-220.ec2.internal" event={"ID":"50b522fb9d3992d5ac1c179e7492fa8c","Type":"ContainerStarted","Data":"1ba69bd9d3ab0e5d8f4f4deece3d432ee80e7e21aa42d63c4abe82078f18ef2d"}
May 25 13:59:54.899915 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.899861    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f-system-cni-dir\") pod \"multus-additional-cni-plugins-msrg9\" (UID: \"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f\") " pod="openshift-multus/multus-additional-cni-plugins-msrg9"
May 25 13:59:54.899915 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.899910    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f-os-release\") pod \"multus-additional-cni-plugins-msrg9\" (UID: \"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f\") " pod="openshift-multus/multus-additional-cni-plugins-msrg9"
May 25 13:59:54.900193 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.899962    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-msrg9\" (UID: \"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f\") " pod="openshift-multus/multus-additional-cni-plugins-msrg9"
May 25 13:59:54.900193 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.899987    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b8d826df-35ec-4d3f-9b5b-eea218065da9-sys-fs\") pod \"aws-ebs-csi-driver-node-926nm\" (UID: \"b8d826df-35ec-4d3f-9b5b-eea218065da9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm"
May 25 13:59:54.900193 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.900018    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f-system-cni-dir\") pod \"multus-additional-cni-plugins-msrg9\" (UID: \"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f\") " pod="openshift-multus/multus-additional-cni-plugins-msrg9"
May 25 13:59:54.900193 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.900021    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b8d826df-35ec-4d3f-9b5b-eea218065da9-device-dir\") pod \"aws-ebs-csi-driver-node-926nm\" (UID: \"b8d826df-35ec-4d3f-9b5b-eea218065da9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm"
May 25 13:59:54.900193 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.900078    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-msrg9\" (UID: \"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f\") " pod="openshift-multus/multus-additional-cni-plugins-msrg9"
May 25 13:59:54.900193 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.900099    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b8d826df-35ec-4d3f-9b5b-eea218065da9-sys-fs\") pod \"aws-ebs-csi-driver-node-926nm\" (UID: \"b8d826df-35ec-4d3f-9b5b-eea218065da9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm"
May 25 13:59:54.900193 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.900104    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f5l7z\" (UniqueName: \"kubernetes.io/projected/370bcb80-f526-48ae-8582-1d104d764c96-kube-api-access-f5l7z\") pod \"node-ca-7jmvx\" (UID: \"370bcb80-f526-48ae-8582-1d104d764c96\") " pod="openshift-image-registry/node-ca-7jmvx"
May 25 13:59:54.900193 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.900110    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f-os-release\") pod \"multus-additional-cni-plugins-msrg9\" (UID: \"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f\") " pod="openshift-multus/multus-additional-cni-plugins-msrg9"
May 25 13:59:54.900193 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.900184    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b8d826df-35ec-4d3f-9b5b-eea218065da9-device-dir\") pod \"aws-ebs-csi-driver-node-926nm\" (UID: \"b8d826df-35ec-4d3f-9b5b-eea218065da9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm"
May 25 13:59:54.900606 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.900265    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-msrg9\" (UID: \"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f\") " pod="openshift-multus/multus-additional-cni-plugins-msrg9"
May 25 13:59:54.900606 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.900337    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f-cni-binary-copy\") pod \"multus-additional-cni-plugins-msrg9\" (UID: \"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f\") " pod="openshift-multus/multus-additional-cni-plugins-msrg9"
May 25 13:59:54.900606 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.900371    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-msrg9\" (UID: \"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f\") " pod="openshift-multus/multus-additional-cni-plugins-msrg9"
May 25 13:59:54.900606 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.900402    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tn656\" (UniqueName: \"kubernetes.io/projected/dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f-kube-api-access-tn656\") pod \"multus-additional-cni-plugins-msrg9\" (UID: \"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f\") " pod="openshift-multus/multus-additional-cni-plugins-msrg9"
May 25 13:59:54.900606 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.900427    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b8d826df-35ec-4d3f-9b5b-eea218065da9-etc-selinux\") pod \"aws-ebs-csi-driver-node-926nm\" (UID: \"b8d826df-35ec-4d3f-9b5b-eea218065da9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm"
May 25 13:59:54.900606 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.900458    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-87mpp\" (UniqueName: \"kubernetes.io/projected/b8d826df-35ec-4d3f-9b5b-eea218065da9-kube-api-access-87mpp\") pod \"aws-ebs-csi-driver-node-926nm\" (UID: \"b8d826df-35ec-4d3f-9b5b-eea218065da9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm"
May 25 13:59:54.900606 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.900489    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f-cnibin\") pod \"multus-additional-cni-plugins-msrg9\" (UID: \"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f\") " pod="openshift-multus/multus-additional-cni-plugins-msrg9"
May 25 13:59:54.900606 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.900520    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b8d826df-35ec-4d3f-9b5b-eea218065da9-socket-dir\") pod \"aws-ebs-csi-driver-node-926nm\" (UID: \"b8d826df-35ec-4d3f-9b5b-eea218065da9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm"
May 25 13:59:54.900606 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.900555    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/370bcb80-f526-48ae-8582-1d104d764c96-serviceca\") pod \"node-ca-7jmvx\" (UID: \"370bcb80-f526-48ae-8582-1d104d764c96\") " pod="openshift-image-registry/node-ca-7jmvx"
May 25 13:59:54.900606 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.900598    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b8d826df-35ec-4d3f-9b5b-eea218065da9-registration-dir\") pod \"aws-ebs-csi-driver-node-926nm\" (UID: \"b8d826df-35ec-4d3f-9b5b-eea218065da9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm"
May 25 13:59:54.901207 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.900611    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-msrg9\" (UID: \"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f\") " pod="openshift-multus/multus-additional-cni-plugins-msrg9"
May 25 13:59:54.901207 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.900628    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b8d826df-35ec-4d3f-9b5b-eea218065da9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-926nm\" (UID: \"b8d826df-35ec-4d3f-9b5b-eea218065da9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm"
May 25 13:59:54.901207 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.900653    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/370bcb80-f526-48ae-8582-1d104d764c96-host\") pod \"node-ca-7jmvx\" (UID: \"370bcb80-f526-48ae-8582-1d104d764c96\") " pod="openshift-image-registry/node-ca-7jmvx"
May 25 13:59:54.901207 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.900655    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f-cnibin\") pod \"multus-additional-cni-plugins-msrg9\" (UID: \"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f\") " pod="openshift-multus/multus-additional-cni-plugins-msrg9"
May 25 13:59:54.901207 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.900795    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b8d826df-35ec-4d3f-9b5b-eea218065da9-etc-selinux\") pod \"aws-ebs-csi-driver-node-926nm\" (UID: \"b8d826df-35ec-4d3f-9b5b-eea218065da9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm"
May 25 13:59:54.901207 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.900880    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f-cni-binary-copy\") pod \"multus-additional-cni-plugins-msrg9\" (UID: \"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f\") " pod="openshift-multus/multus-additional-cni-plugins-msrg9"
May 25 13:59:54.901207 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.900960    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b8d826df-35ec-4d3f-9b5b-eea218065da9-registration-dir\") pod \"aws-ebs-csi-driver-node-926nm\" (UID: \"b8d826df-35ec-4d3f-9b5b-eea218065da9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm"
May 25 13:59:54.901207 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.900964    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b8d826df-35ec-4d3f-9b5b-eea218065da9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-926nm\" (UID: \"b8d826df-35ec-4d3f-9b5b-eea218065da9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm"
May 25 13:59:54.901207 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.900880    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-msrg9\" (UID: \"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f\") " pod="openshift-multus/multus-additional-cni-plugins-msrg9"
May 25 13:59:54.901207 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.901001    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/370bcb80-f526-48ae-8582-1d104d764c96-host\") pod \"node-ca-7jmvx\" (UID: \"370bcb80-f526-48ae-8582-1d104d764c96\") " pod="openshift-image-registry/node-ca-7jmvx"
May 25 13:59:54.901207 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.901024    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b8d826df-35ec-4d3f-9b5b-eea218065da9-socket-dir\") pod \"aws-ebs-csi-driver-node-926nm\" (UID: \"b8d826df-35ec-4d3f-9b5b-eea218065da9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm"
May 25 13:59:54.901207 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.901138    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/370bcb80-f526-48ae-8582-1d104d764c96-serviceca\") pod \"node-ca-7jmvx\" (UID: \"370bcb80-f526-48ae-8582-1d104d764c96\") " pod="openshift-image-registry/node-ca-7jmvx"
May 25 13:59:54.908346 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.908280    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5l7z\" (UniqueName: \"kubernetes.io/projected/370bcb80-f526-48ae-8582-1d104d764c96-kube-api-access-f5l7z\") pod \"node-ca-7jmvx\" (UID: \"370bcb80-f526-48ae-8582-1d104d764c96\") " pod="openshift-image-registry/node-ca-7jmvx"
May 25 13:59:54.908517 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.908491    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87mpp\" (UniqueName: \"kubernetes.io/projected/b8d826df-35ec-4d3f-9b5b-eea218065da9-kube-api-access-87mpp\") pod \"aws-ebs-csi-driver-node-926nm\" (UID: \"b8d826df-35ec-4d3f-9b5b-eea218065da9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm"
May 25 13:59:54.910003 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.909987    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tn656\" (UniqueName: \"kubernetes.io/projected/dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f-kube-api-access-tn656\") pod \"multus-additional-cni-plugins-msrg9\" (UID: \"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f\") " pod="openshift-multus/multus-additional-cni-plugins-msrg9"
May 25 13:59:54.984064 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.984023    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rx92j"
May 25 13:59:54.990979 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:54.990947    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c8rbn"
May 25 13:59:55.001893 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:55.001861    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 13:59:55.006526 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:55.006503    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bw5st"
May 25 13:59:55.022451 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:55.022410    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zm52l"
May 25 13:59:55.031179 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:55.031148    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bt7c9"
May 25 13:59:55.037831 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:55.037805    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm"
May 25 13:59:55.051578 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:55.051543    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7jmvx"
May 25 13:59:55.051733 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:55.051588    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-msrg9"
May 25 13:59:55.303919 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:55.303873    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gs7zp\" (UniqueName: \"kubernetes.io/projected/98f00859-da06-410b-8572-c64348ab32d5-kube-api-access-gs7zp\") pod \"network-check-target-fb2gw\" (UID: \"98f00859-da06-410b-8572-c64348ab32d5\") " pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 13:59:55.304118 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:55.303976    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9f60fa8-15a3-402e-842e-f1a52b7a3404-metrics-certs\") pod \"network-metrics-daemon-lc2jb\" (UID: \"f9f60fa8-15a3-402e-842e-f1a52b7a3404\") " pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 13:59:55.304118 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:55.304064    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 13:59:55.304118 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:55.304073    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 13:59:55.304118 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:55.304089    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 13:59:55.304118 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:55.304099    2571 projected.go:194] Error preparing data for projected volume kube-api-access-gs7zp for pod openshift-network-diagnostics/network-check-target-fb2gw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 13:59:55.304324 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:55.304149    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9f60fa8-15a3-402e-842e-f1a52b7a3404-metrics-certs podName:f9f60fa8-15a3-402e-842e-f1a52b7a3404 nodeName:}" failed. No retries permitted until 2026-05-25 13:59:56.304129861 +0000 UTC m=+4.138985935 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f9f60fa8-15a3-402e-842e-f1a52b7a3404-metrics-certs") pod "network-metrics-daemon-lc2jb" (UID: "f9f60fa8-15a3-402e-842e-f1a52b7a3404") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 13:59:55.304324 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:55.304169    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98f00859-da06-410b-8572-c64348ab32d5-kube-api-access-gs7zp podName:98f00859-da06-410b-8572-c64348ab32d5 nodeName:}" failed. No retries permitted until 2026-05-25 13:59:56.304160301 +0000 UTC m=+4.139016380 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gs7zp" (UniqueName: "kubernetes.io/projected/98f00859-da06-410b-8572-c64348ab32d5-kube-api-access-gs7zp") pod "network-check-target-fb2gw" (UID: "98f00859-da06-410b-8572-c64348ab32d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 13:59:55.606800 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:55.606768    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7a16d3f_06d4_4855_bd37_0e1df89d84b5.slice/crio-701491805b5a9a8748f3ea1cde0c7caf3a5da3f024e46561be9875f517618832 WatchSource:0}: Error finding container 701491805b5a9a8748f3ea1cde0c7caf3a5da3f024e46561be9875f517618832: Status 404 returned error can't find the container with id 701491805b5a9a8748f3ea1cde0c7caf3a5da3f024e46561be9875f517618832
May 25 13:59:55.608426 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:55.608401    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8d826df_35ec_4d3f_9b5b_eea218065da9.slice/crio-4c0ecdd967ed57493fdc4abe8efc3e825dc2b00331585c712922fe80fa7e4b59 WatchSource:0}: Error finding container 4c0ecdd967ed57493fdc4abe8efc3e825dc2b00331585c712922fe80fa7e4b59: Status 404 returned error can't find the container with id 4c0ecdd967ed57493fdc4abe8efc3e825dc2b00331585c712922fe80fa7e4b59
May 25 13:59:55.609156 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:55.609130    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3d0b072_8bba_48c2_98cc_d14341527f7c.slice/crio-1000537239c71f0d2ba907462b7b0ab1ca294f8cad473812a382892e912bab74 WatchSource:0}: Error finding container 1000537239c71f0d2ba907462b7b0ab1ca294f8cad473812a382892e912bab74: Status 404 returned error can't find the container with id 1000537239c71f0d2ba907462b7b0ab1ca294f8cad473812a382892e912bab74
May 25 13:59:55.612400 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:55.612317    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod198a8047_e88d_4bc2_805d_0bfd6155fb0f.slice/crio-c4ae38bd1d25fae542e978d2c7dd2ec66f38880f7335553f93425f4ea3955313 WatchSource:0}: Error finding container c4ae38bd1d25fae542e978d2c7dd2ec66f38880f7335553f93425f4ea3955313: Status 404 returned error can't find the container with id c4ae38bd1d25fae542e978d2c7dd2ec66f38880f7335553f93425f4ea3955313
May 25 13:59:55.619170 ip-10-0-128-220 kubenswrapper[2571]: W0525 13:59:55.619144    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0a2025d_c91e_4fd7_a82a_37a9416c9fc5.slice/crio-589024604ff06ef33f81c6cb59706855863c0945ada07d0e72a1c884beb0ab0c WatchSource:0}: Error finding container 589024604ff06ef33f81c6cb59706855863c0945ada07d0e72a1c884beb0ab0c: Status 404 returned error can't find the container with id 589024604ff06ef33f81c6cb59706855863c0945ada07d0e72a1c884beb0ab0c
May 25 13:59:55.723003 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:55.722958    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-24 13:54:53 +0000 UTC" deadline="2027-11-01 13:26:37.96343804 +0000 UTC"
May 25 13:59:55.723003 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:55.723000    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12599h26m42.240441863s"
May 25 13:59:55.821983 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:55.821952    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 13:59:55.822135 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:55.822103    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc2jb" podUID="f9f60fa8-15a3-402e-842e-f1a52b7a3404"
May 25 13:59:55.829749 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:55.829714    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rx92j" event={"ID":"68251ce2-60d3-4c72-8e3f-58d0ea9d9629","Type":"ContainerStarted","Data":"43b9aac6359c4acabbb8a5105eee2d427eeadff6f69755cedc8f730409dd511d"}
May 25 13:59:55.830665 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:55.830638    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bt7c9" event={"ID":"198a8047-e88d-4bc2-805d-0bfd6155fb0f","Type":"ContainerStarted","Data":"c4ae38bd1d25fae542e978d2c7dd2ec66f38880f7335553f93425f4ea3955313"}
May 25 13:59:55.831690 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:55.831666    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bw5st" event={"ID":"e3d0b072-8bba-48c2-98cc-d14341527f7c","Type":"ContainerStarted","Data":"1000537239c71f0d2ba907462b7b0ab1ca294f8cad473812a382892e912bab74"}
May 25 13:59:55.832620 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:55.832588    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c8rbn" event={"ID":"a0a2025d-c91e-4fd7-a82a-37a9416c9fc5","Type":"ContainerStarted","Data":"589024604ff06ef33f81c6cb59706855863c0945ada07d0e72a1c884beb0ab0c"}
May 25 13:59:55.833569 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:55.833537    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-msrg9" event={"ID":"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f","Type":"ContainerStarted","Data":"8f4a8fd97743896e0aa2e8f4cbc5da7b44d9906774749f097cc41997df6aab09"}
May 25 13:59:55.834475 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:55.834454    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zm52l" event={"ID":"6e393b7c-9ac9-4848-95f7-7aa7845d67d9","Type":"ContainerStarted","Data":"72a980497a7f98da1525b29a479b9b0b8187fb7971e85a7a299d521a20f12692"}
May 25 13:59:55.835386 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:55.835368    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7jmvx" event={"ID":"370bcb80-f526-48ae-8582-1d104d764c96","Type":"ContainerStarted","Data":"6c16913dad0a60f07c09060a6fc62d1a8cf5ea48956a013382dfc9d9c8349970"}
May 25 13:59:55.836265 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:55.836246    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm" event={"ID":"b8d826df-35ec-4d3f-9b5b-eea218065da9","Type":"ContainerStarted","Data":"4c0ecdd967ed57493fdc4abe8efc3e825dc2b00331585c712922fe80fa7e4b59"}
May 25 13:59:55.837191 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:55.837171    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fstv5" event={"ID":"c7a16d3f-06d4-4855-bd37-0e1df89d84b5","Type":"ContainerStarted","Data":"701491805b5a9a8748f3ea1cde0c7caf3a5da3f024e46561be9875f517618832"}
May 25 13:59:55.838588 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:55.838564    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-220.ec2.internal" event={"ID":"9707757169265407340417083d330c50","Type":"ContainerStarted","Data":"3c9a10335173dada69efb8202bd61fe672b5d691e2e9c3e9afd617a61be8a527"}
May 25 13:59:55.852748 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:55.852636    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-220.ec2.internal" podStartSLOduration=1.852617536 podStartE2EDuration="1.852617536s" podCreationTimestamp="2026-05-25 13:59:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 13:59:55.852211266 +0000 UTC m=+3.687067353" watchObservedRunningTime="2026-05-25 13:59:55.852617536 +0000 UTC m=+3.687473630"
May 25 13:59:56.312096 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:56.312053    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gs7zp\" (UniqueName: \"kubernetes.io/projected/98f00859-da06-410b-8572-c64348ab32d5-kube-api-access-gs7zp\") pod \"network-check-target-fb2gw\" (UID: \"98f00859-da06-410b-8572-c64348ab32d5\") " pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 13:59:56.312306 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:56.312134    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9f60fa8-15a3-402e-842e-f1a52b7a3404-metrics-certs\") pod \"network-metrics-daemon-lc2jb\" (UID: \"f9f60fa8-15a3-402e-842e-f1a52b7a3404\") " pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 13:59:56.312306 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:56.312271    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 13:59:56.312428 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:56.312337    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9f60fa8-15a3-402e-842e-f1a52b7a3404-metrics-certs podName:f9f60fa8-15a3-402e-842e-f1a52b7a3404 nodeName:}" failed. No retries permitted until 2026-05-25 13:59:58.312316918 +0000 UTC m=+6.147172994 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f9f60fa8-15a3-402e-842e-f1a52b7a3404-metrics-certs") pod "network-metrics-daemon-lc2jb" (UID: "f9f60fa8-15a3-402e-842e-f1a52b7a3404") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 13:59:56.312491 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:56.312444    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 13:59:56.312491 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:56.312458    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 13:59:56.312491 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:56.312470    2571 projected.go:194] Error preparing data for projected volume kube-api-access-gs7zp for pod openshift-network-diagnostics/network-check-target-fb2gw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 13:59:56.312639 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:56.312508    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98f00859-da06-410b-8572-c64348ab32d5-kube-api-access-gs7zp podName:98f00859-da06-410b-8572-c64348ab32d5 nodeName:}" failed. No retries permitted until 2026-05-25 13:59:58.312496275 +0000 UTC m=+6.147352348 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gs7zp" (UniqueName: "kubernetes.io/projected/98f00859-da06-410b-8572-c64348ab32d5-kube-api-access-gs7zp") pod "network-check-target-fb2gw" (UID: "98f00859-da06-410b-8572-c64348ab32d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 13:59:56.824484 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:56.824442    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 13:59:56.831080 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:56.827302    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2gw" podUID="98f00859-da06-410b-8572-c64348ab32d5"
May 25 13:59:56.848604 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:56.848547    2571 generic.go:358] "Generic (PLEG): container finished" podID="50b522fb9d3992d5ac1c179e7492fa8c" containerID="51d38d2757150ced152cc6d650cd6d9a34bddd0c0e91505c65bc03f94337aec5" exitCode=0
May 25 13:59:56.849705 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:56.849639    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-220.ec2.internal" event={"ID":"50b522fb9d3992d5ac1c179e7492fa8c","Type":"ContainerDied","Data":"51d38d2757150ced152cc6d650cd6d9a34bddd0c0e91505c65bc03f94337aec5"}
May 25 13:59:57.823596 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:57.823562    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 13:59:57.823829 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:57.823712    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc2jb" podUID="f9f60fa8-15a3-402e-842e-f1a52b7a3404"
May 25 13:59:57.855349 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:57.855275    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-220.ec2.internal" event={"ID":"50b522fb9d3992d5ac1c179e7492fa8c","Type":"ContainerStarted","Data":"c06284a22367d3631e56f7445dbdc21ce5b7567372ceaf78360f02f8e1253926"}
May 25 13:59:57.883061 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:57.883000    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-220.ec2.internal" podStartSLOduration=3.88297756 podStartE2EDuration="3.88297756s" podCreationTimestamp="2026-05-25 13:59:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 13:59:57.881903939 +0000 UTC m=+5.716760032" watchObservedRunningTime="2026-05-25 13:59:57.88297756 +0000 UTC m=+5.717833655"
May 25 13:59:58.333375 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:58.333321    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gs7zp\" (UniqueName: \"kubernetes.io/projected/98f00859-da06-410b-8572-c64348ab32d5-kube-api-access-gs7zp\") pod \"network-check-target-fb2gw\" (UID: \"98f00859-da06-410b-8572-c64348ab32d5\") " pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 13:59:58.333587 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:58.333419    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9f60fa8-15a3-402e-842e-f1a52b7a3404-metrics-certs\") pod \"network-metrics-daemon-lc2jb\" (UID: \"f9f60fa8-15a3-402e-842e-f1a52b7a3404\") " pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 13:59:58.333587 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:58.333503    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 13:59:58.333587 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:58.333530    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 13:59:58.333587 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:58.333544    2571 projected.go:194] Error preparing data for projected volume kube-api-access-gs7zp for pod openshift-network-diagnostics/network-check-target-fb2gw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 13:59:58.333587 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:58.333555    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 13:59:58.333836 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:58.333611    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9f60fa8-15a3-402e-842e-f1a52b7a3404-metrics-certs podName:f9f60fa8-15a3-402e-842e-f1a52b7a3404 nodeName:}" failed. No retries permitted until 2026-05-25 14:00:02.333592298 +0000 UTC m=+10.168448372 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f9f60fa8-15a3-402e-842e-f1a52b7a3404-metrics-certs") pod "network-metrics-daemon-lc2jb" (UID: "f9f60fa8-15a3-402e-842e-f1a52b7a3404") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 13:59:58.333836 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:58.333630    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98f00859-da06-410b-8572-c64348ab32d5-kube-api-access-gs7zp podName:98f00859-da06-410b-8572-c64348ab32d5 nodeName:}" failed. No retries permitted until 2026-05-25 14:00:02.333620105 +0000 UTC m=+10.168476180 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gs7zp" (UniqueName: "kubernetes.io/projected/98f00859-da06-410b-8572-c64348ab32d5-kube-api-access-gs7zp") pod "network-check-target-fb2gw" (UID: "98f00859-da06-410b-8572-c64348ab32d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 13:59:58.824439 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:58.824408    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 13:59:58.824616 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:58.824535    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2gw" podUID="98f00859-da06-410b-8572-c64348ab32d5"
May 25 13:59:59.822138 ip-10-0-128-220 kubenswrapper[2571]: I0525 13:59:59.822100    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 13:59:59.822592 ip-10-0-128-220 kubenswrapper[2571]: E0525 13:59:59.822277    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc2jb" podUID="f9f60fa8-15a3-402e-842e-f1a52b7a3404"
May 25 14:00:00.821863 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:00.821830    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 14:00:00.822075 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:00.821977    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2gw" podUID="98f00859-da06-410b-8572-c64348ab32d5"
May 25 14:00:01.822527 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:01.821946    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 14:00:01.822527 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:01.822116    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc2jb" podUID="f9f60fa8-15a3-402e-842e-f1a52b7a3404"
May 25 14:00:02.368057 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:02.368010    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9f60fa8-15a3-402e-842e-f1a52b7a3404-metrics-certs\") pod \"network-metrics-daemon-lc2jb\" (UID: \"f9f60fa8-15a3-402e-842e-f1a52b7a3404\") " pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 14:00:02.368343 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:02.368096    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gs7zp\" (UniqueName: \"kubernetes.io/projected/98f00859-da06-410b-8572-c64348ab32d5-kube-api-access-gs7zp\") pod \"network-check-target-fb2gw\" (UID: \"98f00859-da06-410b-8572-c64348ab32d5\") " pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 14:00:02.368343 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:02.368140    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 14:00:02.368343 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:02.368200    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 14:00:02.368343 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:02.368207    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9f60fa8-15a3-402e-842e-f1a52b7a3404-metrics-certs podName:f9f60fa8-15a3-402e-842e-f1a52b7a3404 nodeName:}" failed. No retries permitted until 2026-05-25 14:00:10.368187425 +0000 UTC m=+18.203043510 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f9f60fa8-15a3-402e-842e-f1a52b7a3404-metrics-certs") pod "network-metrics-daemon-lc2jb" (UID: "f9f60fa8-15a3-402e-842e-f1a52b7a3404") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 14:00:02.368343 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:02.368214    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 14:00:02.368343 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:02.368228    2571 projected.go:194] Error preparing data for projected volume kube-api-access-gs7zp for pod openshift-network-diagnostics/network-check-target-fb2gw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 14:00:02.368343 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:02.368263    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98f00859-da06-410b-8572-c64348ab32d5-kube-api-access-gs7zp podName:98f00859-da06-410b-8572-c64348ab32d5 nodeName:}" failed. No retries permitted until 2026-05-25 14:00:10.368251466 +0000 UTC m=+18.203107537 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gs7zp" (UniqueName: "kubernetes.io/projected/98f00859-da06-410b-8572-c64348ab32d5-kube-api-access-gs7zp") pod "network-check-target-fb2gw" (UID: "98f00859-da06-410b-8572-c64348ab32d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 14:00:02.823250 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:02.823211    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 14:00:02.823679 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:02.823335    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2gw" podUID="98f00859-da06-410b-8572-c64348ab32d5"
May 25 14:00:03.821834 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:03.821635    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 14:00:03.822000 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:03.821945    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc2jb" podUID="f9f60fa8-15a3-402e-842e-f1a52b7a3404"
May 25 14:00:04.822089 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:04.822056    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 14:00:04.822507 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:04.822186    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2gw" podUID="98f00859-da06-410b-8572-c64348ab32d5"
May 25 14:00:05.822185 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:05.822148    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 14:00:05.822543 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:05.822300    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc2jb" podUID="f9f60fa8-15a3-402e-842e-f1a52b7a3404"
May 25 14:00:06.822451 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:06.822411    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 14:00:06.822918 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:06.822552    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2gw" podUID="98f00859-da06-410b-8572-c64348ab32d5"
May 25 14:00:07.822118 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:07.822079    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 14:00:07.822322 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:07.822226    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc2jb" podUID="f9f60fa8-15a3-402e-842e-f1a52b7a3404"
May 25 14:00:08.821780 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:08.821738    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 14:00:08.822265 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:08.821877    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2gw" podUID="98f00859-da06-410b-8572-c64348ab32d5"
May 25 14:00:09.822304 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:09.822264    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 14:00:09.822787 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:09.822415    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc2jb" podUID="f9f60fa8-15a3-402e-842e-f1a52b7a3404"
May 25 14:00:10.432732 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:10.432686    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gs7zp\" (UniqueName: \"kubernetes.io/projected/98f00859-da06-410b-8572-c64348ab32d5-kube-api-access-gs7zp\") pod \"network-check-target-fb2gw\" (UID: \"98f00859-da06-410b-8572-c64348ab32d5\") " pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 14:00:10.432919 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:10.432778    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9f60fa8-15a3-402e-842e-f1a52b7a3404-metrics-certs\") pod \"network-metrics-daemon-lc2jb\" (UID: \"f9f60fa8-15a3-402e-842e-f1a52b7a3404\") " pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 14:00:10.432919 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:10.432855    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 14:00:10.432919 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:10.432888    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 14:00:10.432919 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:10.432898    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 14:00:10.432919 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:10.432902    2571 projected.go:194] Error preparing data for projected volume kube-api-access-gs7zp for pod openshift-network-diagnostics/network-check-target-fb2gw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 14:00:10.433150 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:10.432977    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9f60fa8-15a3-402e-842e-f1a52b7a3404-metrics-certs podName:f9f60fa8-15a3-402e-842e-f1a52b7a3404 nodeName:}" failed. No retries permitted until 2026-05-25 14:00:26.432954642 +0000 UTC m=+34.267810714 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f9f60fa8-15a3-402e-842e-f1a52b7a3404-metrics-certs") pod "network-metrics-daemon-lc2jb" (UID: "f9f60fa8-15a3-402e-842e-f1a52b7a3404") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 14:00:10.433150 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:10.433000    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98f00859-da06-410b-8572-c64348ab32d5-kube-api-access-gs7zp podName:98f00859-da06-410b-8572-c64348ab32d5 nodeName:}" failed. No retries permitted until 2026-05-25 14:00:26.432987468 +0000 UTC m=+34.267843539 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gs7zp" (UniqueName: "kubernetes.io/projected/98f00859-da06-410b-8572-c64348ab32d5-kube-api-access-gs7zp") pod "network-check-target-fb2gw" (UID: "98f00859-da06-410b-8572-c64348ab32d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 14:00:10.821994 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:10.821955    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 14:00:10.822161 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:10.822110    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2gw" podUID="98f00859-da06-410b-8572-c64348ab32d5"
May 25 14:00:11.821960 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:11.821729    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 14:00:11.821960 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:11.821872    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc2jb" podUID="f9f60fa8-15a3-402e-842e-f1a52b7a3404"
May 25 14:00:12.823296 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:12.823251    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 14:00:12.823872 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:12.823369    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2gw" podUID="98f00859-da06-410b-8572-c64348ab32d5"
May 25 14:00:13.821383 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:13.821347    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 14:00:13.821557 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:13.821468    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc2jb" podUID="f9f60fa8-15a3-402e-842e-f1a52b7a3404"
May 25 14:00:14.821807 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:14.821637    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 14:00:14.822378 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:14.821885    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2gw" podUID="98f00859-da06-410b-8572-c64348ab32d5"
May 25 14:00:14.892521 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:14.892483    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c8rbn" event={"ID":"a0a2025d-c91e-4fd7-a82a-37a9416c9fc5","Type":"ContainerStarted","Data":"83dd6a9bec5aca4cf9882212d037b7977c088e6ad0e7c9d5a5b3e022439f55eb"}
May 25 14:00:14.894153 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:14.894126    2571 generic.go:358] "Generic (PLEG): container finished" podID="dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f" containerID="bc14b60b516c3972100fc287e68b78849d5dbc7d01bd7d6e108947823c1c4674" exitCode=0
May 25 14:00:14.894284 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:14.894188    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-msrg9" event={"ID":"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f","Type":"ContainerDied","Data":"bc14b60b516c3972100fc287e68b78849d5dbc7d01bd7d6e108947823c1c4674"}
May 25 14:00:14.895615 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:14.895588    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zm52l" event={"ID":"6e393b7c-9ac9-4848-95f7-7aa7845d67d9","Type":"ContainerStarted","Data":"6a18c62a2b82962fc1625ae9f903a6f4bae4ac3de3942c1e4f35a8bedc3238d0"}
May 25 14:00:14.896982 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:14.896951    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7jmvx" event={"ID":"370bcb80-f526-48ae-8582-1d104d764c96","Type":"ContainerStarted","Data":"d68295ce921b6467e75adfede27f91a6b491ed4669c00d392d452f16d6ac03b9"}
May 25 14:00:14.898294 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:14.898268    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm" event={"ID":"b8d826df-35ec-4d3f-9b5b-eea218065da9","Type":"ContainerStarted","Data":"81cdaed8341e5cca7c95a9fca6a03efe85741c139b5ef1ed01e57bee4e12e692"}
May 25 14:00:14.903540 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:14.903509    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fstv5" event={"ID":"c7a16d3f-06d4-4855-bd37-0e1df89d84b5","Type":"ContainerStarted","Data":"324f729b0d19532217a95d74fad8e3d496ae2ed01924d889fcc82c5bc66d862f"}
May 25 14:00:14.903663 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:14.903544    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fstv5" event={"ID":"c7a16d3f-06d4-4855-bd37-0e1df89d84b5","Type":"ContainerStarted","Data":"9c516c2c3e4debe21578e4f59aec95c61218c7424d2000ec47119a2ee6948b7b"}
May 25 14:00:14.903663 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:14.903558    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fstv5" event={"ID":"c7a16d3f-06d4-4855-bd37-0e1df89d84b5","Type":"ContainerStarted","Data":"56885375a91d1d2d86cbaee076d34e466c9522ca36517695fa1f9b9de385f140"}
May 25 14:00:14.903663 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:14.903571    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fstv5" event={"ID":"c7a16d3f-06d4-4855-bd37-0e1df89d84b5","Type":"ContainerStarted","Data":"e80f5620401fd8255fa0cfc63ac7d6ce9f2418e32f82ecfaa53b7055de055fad"}
May 25 14:00:14.903663 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:14.903587    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fstv5" event={"ID":"c7a16d3f-06d4-4855-bd37-0e1df89d84b5","Type":"ContainerStarted","Data":"ac437260d8ac1dd2e803a5d40efff6693a1425beb08a7f46b1057756260c5123"}
May 25 14:00:14.903663 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:14.903599    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fstv5" event={"ID":"c7a16d3f-06d4-4855-bd37-0e1df89d84b5","Type":"ContainerStarted","Data":"495ef2f67f546442c43bbc489d415a4fadbbc7536e32f73c0933a7b66e5db613"}
May 25 14:00:14.904909 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:14.904748    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rx92j" event={"ID":"68251ce2-60d3-4c72-8e3f-58d0ea9d9629","Type":"ContainerStarted","Data":"2129e230b37baa9e3d394eaa212894e643cf61fe8d69e468cb9fed777fa1ffd4"}
May 25 14:00:14.905916 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:14.905890    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bw5st" event={"ID":"e3d0b072-8bba-48c2-98cc-d14341527f7c","Type":"ContainerStarted","Data":"ed674b24ce995f22205e7814c73c1fbedc97a2a02caa67ea5ec6d78cdf9bb9b9"}
May 25 14:00:14.911160 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:14.911120    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-c8rbn" podStartSLOduration=4.618959859 podStartE2EDuration="22.911109025s" podCreationTimestamp="2026-05-25 13:59:52 +0000 UTC" firstStartedPulling="2026-05-25 13:59:55.620694042 +0000 UTC m=+3.455550128" lastFinishedPulling="2026-05-25 14:00:13.912843204 +0000 UTC m=+21.747699294" observedRunningTime="2026-05-25 14:00:14.910640151 +0000 UTC m=+22.745496235" watchObservedRunningTime="2026-05-25 14:00:14.911109025 +0000 UTC m=+22.745965118"
May 25 14:00:14.944740 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:14.944684    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-rx92j" podStartSLOduration=4.2541715159999995 podStartE2EDuration="22.94466833s" podCreationTimestamp="2026-05-25 13:59:52 +0000 UTC" firstStartedPulling="2026-05-25 13:59:55.616897145 +0000 UTC m=+3.451753220" lastFinishedPulling="2026-05-25 14:00:14.307393963 +0000 UTC m=+22.142250034" observedRunningTime="2026-05-25 14:00:14.944192793 +0000 UTC m=+22.779048887" watchObservedRunningTime="2026-05-25 14:00:14.94466833 +0000 UTC m=+22.779524422"
May 25 14:00:14.960360 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:14.960310    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-zm52l" podStartSLOduration=3.213084614 podStartE2EDuration="21.960295454s" podCreationTimestamp="2026-05-25 13:59:53 +0000 UTC" firstStartedPulling="2026-05-25 13:59:55.617146975 +0000 UTC m=+3.452003058" lastFinishedPulling="2026-05-25 14:00:14.364357812 +0000 UTC m=+22.199213898" observedRunningTime="2026-05-25 14:00:14.960175922 +0000 UTC m=+22.795032015" watchObservedRunningTime="2026-05-25 14:00:14.960295454 +0000 UTC m=+22.795151546"
May 25 14:00:14.976174 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:14.976122    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bw5st" podStartSLOduration=3.2805745809999998 podStartE2EDuration="21.976107046s" podCreationTimestamp="2026-05-25 13:59:53 +0000 UTC" firstStartedPulling="2026-05-25 13:59:55.611818772 +0000 UTC m=+3.446674844" lastFinishedPulling="2026-05-25 14:00:14.307351223 +0000 UTC m=+22.142207309" observedRunningTime="2026-05-25 14:00:14.976035973 +0000 UTC m=+22.810892067" watchObservedRunningTime="2026-05-25 14:00:14.976107046 +0000 UTC m=+22.810963138"
May 25 14:00:14.989008 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:14.988959    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7jmvx" podStartSLOduration=3.339386467 podStartE2EDuration="21.988914655s" podCreationTimestamp="2026-05-25 13:59:53 +0000 UTC" firstStartedPulling="2026-05-25 13:59:55.615741558 +0000 UTC m=+3.450597628" lastFinishedPulling="2026-05-25 14:00:14.265269745 +0000 UTC m=+22.100125816" observedRunningTime="2026-05-25 14:00:14.988889493 +0000 UTC m=+22.823745585" watchObservedRunningTime="2026-05-25 14:00:14.988914655 +0000 UTC m=+22.823770747"
May 25 14:00:15.597396 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:15.597210    2571 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 25 14:00:15.762179 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:15.762076    2571 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-25T14:00:15.597391635Z","UUID":"e25413c0-3f0b-4669-a29b-46f9804b0ffc","Handler":null,"Name":"","Endpoint":""}
May 25 14:00:15.765222 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:15.765201    2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 25 14:00:15.765222 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:15.765229    2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 25 14:00:15.822012 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:15.821978    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 14:00:15.822526 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:15.822105    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc2jb" podUID="f9f60fa8-15a3-402e-842e-f1a52b7a3404"
May 25 14:00:15.911053 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:15.910957    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bt7c9" event={"ID":"198a8047-e88d-4bc2-805d-0bfd6155fb0f","Type":"ContainerStarted","Data":"3e50cde1bef497c0492afd3f576de2f03602b8de3392545d33403d83ff75232b"}
May 25 14:00:15.912842 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:15.912812    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm" event={"ID":"b8d826df-35ec-4d3f-9b5b-eea218065da9","Type":"ContainerStarted","Data":"0254656fea5837c030117d025ac72848142fc83aa039a3767e1749fc7ab1314c"}
May 25 14:00:15.924730 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:15.924677    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bt7c9" podStartSLOduration=4.231956822 podStartE2EDuration="22.924660999s" podCreationTimestamp="2026-05-25 13:59:53 +0000 UTC" firstStartedPulling="2026-05-25 13:59:55.615659669 +0000 UTC m=+3.450515740" lastFinishedPulling="2026-05-25 14:00:14.308363833 +0000 UTC m=+22.143219917" observedRunningTime="2026-05-25 14:00:15.924294336 +0000 UTC m=+23.759150431" watchObservedRunningTime="2026-05-25 14:00:15.924660999 +0000 UTC m=+23.759517092"
May 25 14:00:16.821531 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:16.821423    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 14:00:16.821690 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:16.821580    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2gw" podUID="98f00859-da06-410b-8572-c64348ab32d5"
May 25 14:00:16.918559 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:16.918524    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm" event={"ID":"b8d826df-35ec-4d3f-9b5b-eea218065da9","Type":"ContainerStarted","Data":"e272d5ec0799c1517a3639f78e97e245a4ed2a406683b3b1e33cecc7182baf6f"}
May 25 14:00:16.936542 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:16.936493    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-926nm" podStartSLOduration=3.011065724 podStartE2EDuration="23.936478551s" podCreationTimestamp="2026-05-25 13:59:53 +0000 UTC" firstStartedPulling="2026-05-25 13:59:55.610975938 +0000 UTC m=+3.445832026" lastFinishedPulling="2026-05-25 14:00:16.53638877 +0000 UTC m=+24.371244853" observedRunningTime="2026-05-25 14:00:16.936103967 +0000 UTC m=+24.770960054" watchObservedRunningTime="2026-05-25 14:00:16.936478551 +0000 UTC m=+24.771334644"
May 25 14:00:17.438575 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:17.438537    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-c8rbn"
May 25 14:00:17.439279 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:17.439254    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-c8rbn"
May 25 14:00:17.821429 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:17.821342    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 14:00:17.821605 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:17.821492    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc2jb" podUID="f9f60fa8-15a3-402e-842e-f1a52b7a3404"
May 25 14:00:17.924300 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:17.924256    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fstv5" event={"ID":"c7a16d3f-06d4-4855-bd37-0e1df89d84b5","Type":"ContainerStarted","Data":"24c6d4e35a4c57039f9f0ba2972f43ac2569b02400ef8dbd04ddc92aca1cff84"}
May 25 14:00:18.821875 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:18.821837    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 14:00:18.822096 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:18.821997    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2gw" podUID="98f00859-da06-410b-8572-c64348ab32d5"
May 25 14:00:19.822387 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:19.822351    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 14:00:19.822860 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:19.822494    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc2jb" podUID="f9f60fa8-15a3-402e-842e-f1a52b7a3404"
May 25 14:00:19.929318 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:19.929227    2571 generic.go:358] "Generic (PLEG): container finished" podID="dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f" containerID="c868962c5350cbd5e481ae1b1b6e055c9664012d1983e72a66b13efd85a66b33" exitCode=0
May 25 14:00:19.929318 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:19.929301    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-msrg9" event={"ID":"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f","Type":"ContainerDied","Data":"c868962c5350cbd5e481ae1b1b6e055c9664012d1983e72a66b13efd85a66b33"}
May 25 14:00:19.932663 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:19.932626    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fstv5" event={"ID":"c7a16d3f-06d4-4855-bd37-0e1df89d84b5","Type":"ContainerStarted","Data":"9cab9e968b75b7d61687206d5ca3b435538d93894dc28ae4b22ec587e3d33d31"}
May 25 14:00:19.932924 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:19.932902    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 14:00:19.933019 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:19.932958    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 14:00:19.933019 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:19.932973    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 14:00:19.952260 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:19.952229    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 14:00:19.952391 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:19.952315    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 14:00:19.974233 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:19.974183    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fstv5" podStartSLOduration=9.203628118 podStartE2EDuration="27.974168384s" podCreationTimestamp="2026-05-25 13:59:52 +0000 UTC" firstStartedPulling="2026-05-25 13:59:55.608629501 +0000 UTC m=+3.443485572" lastFinishedPulling="2026-05-25 14:00:14.379169751 +0000 UTC m=+22.214025838" observedRunningTime="2026-05-25 14:00:19.973803157 +0000 UTC m=+27.808659249" watchObservedRunningTime="2026-05-25 14:00:19.974168384 +0000 UTC m=+27.809024477"
May 25 14:00:20.821989 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:20.821951    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 14:00:20.822184 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:20.822084    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2gw" podUID="98f00859-da06-410b-8572-c64348ab32d5"
May 25 14:00:21.822130 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:21.821949    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 14:00:21.822475 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:21.822223    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc2jb" podUID="f9f60fa8-15a3-402e-842e-f1a52b7a3404"
May 25 14:00:21.937862 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:21.937823    2571 generic.go:358] "Generic (PLEG): container finished" podID="dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f" containerID="584c174b2ac7aa5ef7bec869f608c1128f321c30e9dd509cae63763c860edee6" exitCode=0
May 25 14:00:21.938070 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:21.937869    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-msrg9" event={"ID":"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f","Type":"ContainerDied","Data":"584c174b2ac7aa5ef7bec869f608c1128f321c30e9dd509cae63763c860edee6"}
May 25 14:00:22.584154 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:22.584118    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fb2gw"]
May 25 14:00:22.584335 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:22.584260    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 14:00:22.584397 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:22.584373    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2gw" podUID="98f00859-da06-410b-8572-c64348ab32d5"
May 25 14:00:22.587147 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:22.587119    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lc2jb"]
May 25 14:00:22.587258 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:22.587233    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 14:00:22.587364 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:22.587347    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc2jb" podUID="f9f60fa8-15a3-402e-842e-f1a52b7a3404"
May 25 14:00:23.343388 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:23.343297    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-c8rbn"
May 25 14:00:23.344344 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:23.343474    2571 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
May 25 14:00:23.344344 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:23.343989    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-c8rbn"
May 25 14:00:23.821869 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:23.821837    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 14:00:23.822036 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:23.821959    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2gw" podUID="98f00859-da06-410b-8572-c64348ab32d5"
May 25 14:00:23.944442 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:23.944411    2571 generic.go:358] "Generic (PLEG): container finished" podID="dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f" containerID="7a1642f2c19c8d493ee76c50e312beeb9ab57933935b33b28d2de3a5ece90ff6" exitCode=0
May 25 14:00:23.944621 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:23.944503    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-msrg9" event={"ID":"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f","Type":"ContainerDied","Data":"7a1642f2c19c8d493ee76c50e312beeb9ab57933935b33b28d2de3a5ece90ff6"}
May 25 14:00:24.824970 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:24.824915    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 14:00:24.825410 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:24.825077    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc2jb" podUID="f9f60fa8-15a3-402e-842e-f1a52b7a3404"
May 25 14:00:25.822053 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:25.821850    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 14:00:25.822313 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:25.822131    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2gw" podUID="98f00859-da06-410b-8572-c64348ab32d5"
May 25 14:00:26.451554 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:26.451516    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gs7zp\" (UniqueName: \"kubernetes.io/projected/98f00859-da06-410b-8572-c64348ab32d5-kube-api-access-gs7zp\") pod \"network-check-target-fb2gw\" (UID: \"98f00859-da06-410b-8572-c64348ab32d5\") " pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 14:00:26.452008 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:26.451574    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9f60fa8-15a3-402e-842e-f1a52b7a3404-metrics-certs\") pod \"network-metrics-daemon-lc2jb\" (UID: \"f9f60fa8-15a3-402e-842e-f1a52b7a3404\") " pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 14:00:26.452008 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:26.451684    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 14:00:26.452008 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:26.451699    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 14:00:26.452008 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:26.451724    2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 14:00:26.452008 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:26.451737    2571 projected.go:194] Error preparing data for projected volume kube-api-access-gs7zp for pod openshift-network-diagnostics/network-check-target-fb2gw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 14:00:26.452008 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:26.451747    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9f60fa8-15a3-402e-842e-f1a52b7a3404-metrics-certs podName:f9f60fa8-15a3-402e-842e-f1a52b7a3404 nodeName:}" failed. No retries permitted until 2026-05-25 14:00:58.451726729 +0000 UTC m=+66.286582801 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f9f60fa8-15a3-402e-842e-f1a52b7a3404-metrics-certs") pod "network-metrics-daemon-lc2jb" (UID: "f9f60fa8-15a3-402e-842e-f1a52b7a3404") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 14:00:26.452008 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:26.451797    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98f00859-da06-410b-8572-c64348ab32d5-kube-api-access-gs7zp podName:98f00859-da06-410b-8572-c64348ab32d5 nodeName:}" failed. No retries permitted until 2026-05-25 14:00:58.451779972 +0000 UTC m=+66.286636043 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-gs7zp" (UniqueName: "kubernetes.io/projected/98f00859-da06-410b-8572-c64348ab32d5-kube-api-access-gs7zp") pod "network-check-target-fb2gw" (UID: "98f00859-da06-410b-8572-c64348ab32d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 14:00:26.821699 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:26.821610    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 14:00:26.821889 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:26.821763    2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc2jb" podUID="f9f60fa8-15a3-402e-842e-f1a52b7a3404"
May 25 14:00:26.967927 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:26.967895    2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-220.ec2.internal" event="NodeReady"
May 25 14:00:26.968124 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:26.968058    2571 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 25 14:00:27.013238 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.013199    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5kgzp"]
May 25 14:00:27.037297 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.037263    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-sn7fl"]
May 25 14:00:27.037469 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.037450    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5kgzp"
May 25 14:00:27.039910 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.039864    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 25 14:00:27.039910 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.039905    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 25 14:00:27.040173 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.039865    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 25 14:00:27.040173 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.039864    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-b5jq8\""
May 25 14:00:27.040173 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.039869    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 25 14:00:27.055228 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.055204    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5kgzp"]
May 25 14:00:27.055363 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.055233    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sn7fl"]
May 25 14:00:27.055363 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.055341    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sn7fl"
May 25 14:00:27.057402 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.057380    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 25 14:00:27.057586 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.057569    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nl2xp\""
May 25 14:00:27.057636 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.057571    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 25 14:00:27.057636 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.057566    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 25 14:00:27.122886 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.122765    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jqs89"]
May 25 14:00:27.143556 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.143522    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jqs89"]
May 25 14:00:27.143735 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.143675    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jqs89"
May 25 14:00:27.146200 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.146172    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 25 14:00:27.146367 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.146220    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kj95h\""
May 25 14:00:27.146480 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.146466    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 25 14:00:27.158231 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.158194    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/24931dfe-9faa-4088-99c8-84ccb73daad2-crio-socket\") pod \"insights-runtime-extractor-5kgzp\" (UID: \"24931dfe-9faa-4088-99c8-84ccb73daad2\") " pod="openshift-insights/insights-runtime-extractor-5kgzp"
May 25 14:00:27.158419 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.158250    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/678dec39-f2e8-486d-9f09-56a72f3cdd85-cert\") pod \"ingress-canary-sn7fl\" (UID: \"678dec39-f2e8-486d-9f09-56a72f3cdd85\") " pod="openshift-ingress-canary/ingress-canary-sn7fl"
May 25 14:00:27.158419 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.158358    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp8d7\" (UniqueName: \"kubernetes.io/projected/678dec39-f2e8-486d-9f09-56a72f3cdd85-kube-api-access-kp8d7\") pod \"ingress-canary-sn7fl\" (UID: \"678dec39-f2e8-486d-9f09-56a72f3cdd85\") " pod="openshift-ingress-canary/ingress-canary-sn7fl"
May 25 14:00:27.158419 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.158408    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24931dfe-9faa-4088-99c8-84ccb73daad2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5kgzp\" (UID: \"24931dfe-9faa-4088-99c8-84ccb73daad2\") " pod="openshift-insights/insights-runtime-extractor-5kgzp"
May 25 14:00:27.158563 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.158442    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8c6pb\" (UniqueName: \"kubernetes.io/projected/24931dfe-9faa-4088-99c8-84ccb73daad2-kube-api-access-8c6pb\") pod \"insights-runtime-extractor-5kgzp\" (UID: \"24931dfe-9faa-4088-99c8-84ccb73daad2\") " pod="openshift-insights/insights-runtime-extractor-5kgzp"
May 25 14:00:27.158563 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.158478    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/24931dfe-9faa-4088-99c8-84ccb73daad2-data-volume\") pod \"insights-runtime-extractor-5kgzp\" (UID: \"24931dfe-9faa-4088-99c8-84ccb73daad2\") " pod="openshift-insights/insights-runtime-extractor-5kgzp"
May 25 14:00:27.158563 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.158505    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24931dfe-9faa-4088-99c8-84ccb73daad2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5kgzp\" (UID: \"24931dfe-9faa-4088-99c8-84ccb73daad2\") " pod="openshift-insights/insights-runtime-extractor-5kgzp"
May 25 14:00:27.259111 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.259073    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/678dec39-f2e8-486d-9f09-56a72f3cdd85-cert\") pod \"ingress-canary-sn7fl\" (UID: \"678dec39-f2e8-486d-9f09-56a72f3cdd85\") " pod="openshift-ingress-canary/ingress-canary-sn7fl"
May 25 14:00:27.259288 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.259118    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68592d9d-e82d-4b8f-9be9-572afeaf463c-tmp-dir\") pod \"dns-default-jqs89\" (UID: \"68592d9d-e82d-4b8f-9be9-572afeaf463c\") " pod="openshift-dns/dns-default-jqs89"
May 25 14:00:27.259288 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.259156    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/68592d9d-e82d-4b8f-9be9-572afeaf463c-config-volume\") pod \"dns-default-jqs89\" (UID: \"68592d9d-e82d-4b8f-9be9-572afeaf463c\") " pod="openshift-dns/dns-default-jqs89"
May 25 14:00:27.259288 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.259186    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8qmc\" (UniqueName: \"kubernetes.io/projected/68592d9d-e82d-4b8f-9be9-572afeaf463c-kube-api-access-t8qmc\") pod \"dns-default-jqs89\" (UID: \"68592d9d-e82d-4b8f-9be9-572afeaf463c\") " pod="openshift-dns/dns-default-jqs89"
May 25 14:00:27.259288 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.259269    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kp8d7\" (UniqueName: \"kubernetes.io/projected/678dec39-f2e8-486d-9f09-56a72f3cdd85-kube-api-access-kp8d7\") pod \"ingress-canary-sn7fl\" (UID: \"678dec39-f2e8-486d-9f09-56a72f3cdd85\") " pod="openshift-ingress-canary/ingress-canary-sn7fl"
May 25 14:00:27.259475 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.259311    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24931dfe-9faa-4088-99c8-84ccb73daad2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5kgzp\" (UID: \"24931dfe-9faa-4088-99c8-84ccb73daad2\") " pod="openshift-insights/insights-runtime-extractor-5kgzp"
May 25 14:00:27.259475 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.259337    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8c6pb\" (UniqueName: \"kubernetes.io/projected/24931dfe-9faa-4088-99c8-84ccb73daad2-kube-api-access-8c6pb\") pod \"insights-runtime-extractor-5kgzp\" (UID: \"24931dfe-9faa-4088-99c8-84ccb73daad2\") " pod="openshift-insights/insights-runtime-extractor-5kgzp"
May 25 14:00:27.259475 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.259365    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/24931dfe-9faa-4088-99c8-84ccb73daad2-data-volume\") pod \"insights-runtime-extractor-5kgzp\" (UID: \"24931dfe-9faa-4088-99c8-84ccb73daad2\") " pod="openshift-insights/insights-runtime-extractor-5kgzp"
May 25 14:00:27.259475 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.259384    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24931dfe-9faa-4088-99c8-84ccb73daad2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5kgzp\" (UID: \"24931dfe-9faa-4088-99c8-84ccb73daad2\") " pod="openshift-insights/insights-runtime-extractor-5kgzp"
May 25 14:00:27.259475 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.259412    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68592d9d-e82d-4b8f-9be9-572afeaf463c-metrics-tls\") pod \"dns-default-jqs89\" (UID: \"68592d9d-e82d-4b8f-9be9-572afeaf463c\") " pod="openshift-dns/dns-default-jqs89"
May 25 14:00:27.259475 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.259466    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/24931dfe-9faa-4088-99c8-84ccb73daad2-crio-socket\") pod \"insights-runtime-extractor-5kgzp\" (UID: \"24931dfe-9faa-4088-99c8-84ccb73daad2\") " pod="openshift-insights/insights-runtime-extractor-5kgzp"
May 25 14:00:27.259747 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.259669    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/24931dfe-9faa-4088-99c8-84ccb73daad2-crio-socket\") pod \"insights-runtime-extractor-5kgzp\" (UID: \"24931dfe-9faa-4088-99c8-84ccb73daad2\") " pod="openshift-insights/insights-runtime-extractor-5kgzp"
May 25 14:00:27.259747 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.259692    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/24931dfe-9faa-4088-99c8-84ccb73daad2-data-volume\") pod \"insights-runtime-extractor-5kgzp\" (UID: \"24931dfe-9faa-4088-99c8-84ccb73daad2\") " pod="openshift-insights/insights-runtime-extractor-5kgzp"
May 25 14:00:27.260064 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.260043    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24931dfe-9faa-4088-99c8-84ccb73daad2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5kgzp\" (UID: \"24931dfe-9faa-4088-99c8-84ccb73daad2\") " pod="openshift-insights/insights-runtime-extractor-5kgzp"
May 25 14:00:27.263418 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.263397    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24931dfe-9faa-4088-99c8-84ccb73daad2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5kgzp\" (UID: \"24931dfe-9faa-4088-99c8-84ccb73daad2\") " pod="openshift-insights/insights-runtime-extractor-5kgzp"
May 25 14:00:27.263622 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.263602    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/678dec39-f2e8-486d-9f09-56a72f3cdd85-cert\") pod \"ingress-canary-sn7fl\" (UID: \"678dec39-f2e8-486d-9f09-56a72f3cdd85\") " pod="openshift-ingress-canary/ingress-canary-sn7fl"
May 25 14:00:27.267905 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.267860    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8c6pb\" (UniqueName: \"kubernetes.io/projected/24931dfe-9faa-4088-99c8-84ccb73daad2-kube-api-access-8c6pb\") pod \"insights-runtime-extractor-5kgzp\" (UID: \"24931dfe-9faa-4088-99c8-84ccb73daad2\") " pod="openshift-insights/insights-runtime-extractor-5kgzp"
May 25 14:00:27.268063 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.268044    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp8d7\" (UniqueName: \"kubernetes.io/projected/678dec39-f2e8-486d-9f09-56a72f3cdd85-kube-api-access-kp8d7\") pod \"ingress-canary-sn7fl\" (UID: \"678dec39-f2e8-486d-9f09-56a72f3cdd85\") " pod="openshift-ingress-canary/ingress-canary-sn7fl"
May 25 14:00:27.355920 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.355884    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5kgzp"
May 25 14:00:27.360093 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.360050    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68592d9d-e82d-4b8f-9be9-572afeaf463c-metrics-tls\") pod \"dns-default-jqs89\" (UID: \"68592d9d-e82d-4b8f-9be9-572afeaf463c\") " pod="openshift-dns/dns-default-jqs89"
May 25 14:00:27.360214 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.360130    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68592d9d-e82d-4b8f-9be9-572afeaf463c-tmp-dir\") pod \"dns-default-jqs89\" (UID: \"68592d9d-e82d-4b8f-9be9-572afeaf463c\") " pod="openshift-dns/dns-default-jqs89"
May 25 14:00:27.360214 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.360159    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/68592d9d-e82d-4b8f-9be9-572afeaf463c-config-volume\") pod \"dns-default-jqs89\" (UID: \"68592d9d-e82d-4b8f-9be9-572afeaf463c\") " pod="openshift-dns/dns-default-jqs89"
May 25 14:00:27.360214 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.360191    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t8qmc\" (UniqueName: \"kubernetes.io/projected/68592d9d-e82d-4b8f-9be9-572afeaf463c-kube-api-access-t8qmc\") pod \"dns-default-jqs89\" (UID: \"68592d9d-e82d-4b8f-9be9-572afeaf463c\") " pod="openshift-dns/dns-default-jqs89"
May 25 14:00:27.360538 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.360517    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68592d9d-e82d-4b8f-9be9-572afeaf463c-tmp-dir\") pod \"dns-default-jqs89\" (UID: \"68592d9d-e82d-4b8f-9be9-572afeaf463c\") " pod="openshift-dns/dns-default-jqs89"
May 25 14:00:27.360807 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.360778    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/68592d9d-e82d-4b8f-9be9-572afeaf463c-config-volume\") pod \"dns-default-jqs89\" (UID: \"68592d9d-e82d-4b8f-9be9-572afeaf463c\") " pod="openshift-dns/dns-default-jqs89"
May 25 14:00:27.362865 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.362833    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sn7fl"
May 25 14:00:27.363141 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.363123    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68592d9d-e82d-4b8f-9be9-572afeaf463c-metrics-tls\") pod \"dns-default-jqs89\" (UID: \"68592d9d-e82d-4b8f-9be9-572afeaf463c\") " pod="openshift-dns/dns-default-jqs89"
May 25 14:00:27.370440 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.370403    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8qmc\" (UniqueName: \"kubernetes.io/projected/68592d9d-e82d-4b8f-9be9-572afeaf463c-kube-api-access-t8qmc\") pod \"dns-default-jqs89\" (UID: \"68592d9d-e82d-4b8f-9be9-572afeaf463c\") " pod="openshift-dns/dns-default-jqs89"
May 25 14:00:27.456773 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.456736    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jqs89"
May 25 14:00:27.539035 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.538995    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5kgzp"]
May 25 14:00:27.543685 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.543587    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sn7fl"]
May 25 14:00:27.632076 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.632032    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jqs89"]
May 25 14:00:27.639092 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:00:27.639060    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68592d9d_e82d_4b8f_9be9_572afeaf463c.slice/crio-065da93adf9fabf09652e5ed2dd40c3941b9c01cc7f5030dc77fc9af138b64c1 WatchSource:0}: Error finding container 065da93adf9fabf09652e5ed2dd40c3941b9c01cc7f5030dc77fc9af138b64c1: Status 404 returned error can't find the container with id 065da93adf9fabf09652e5ed2dd40c3941b9c01cc7f5030dc77fc9af138b64c1
May 25 14:00:27.822261 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.822177    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 14:00:27.824423 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.824396    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 25 14:00:27.824556 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.824422    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 25 14:00:27.824556 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.824396    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zr45f\""
May 25 14:00:27.953881 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.953838    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jqs89" event={"ID":"68592d9d-e82d-4b8f-9be9-572afeaf463c","Type":"ContainerStarted","Data":"065da93adf9fabf09652e5ed2dd40c3941b9c01cc7f5030dc77fc9af138b64c1"}
May 25 14:00:27.955233 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.955204    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sn7fl" event={"ID":"678dec39-f2e8-486d-9f09-56a72f3cdd85","Type":"ContainerStarted","Data":"2db5cdf01606fed09a3c2d068e02370d4e3f664cb8433fbca052d9b787ac6034"}
May 25 14:00:27.956627 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.956595    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5kgzp" event={"ID":"24931dfe-9faa-4088-99c8-84ccb73daad2","Type":"ContainerStarted","Data":"9dcdc8e7f58aab9658d717b7ee4f75c5e632cb7688d436a622cfe85edc2ab42e"}
May 25 14:00:27.956744 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.956634    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5kgzp" event={"ID":"24931dfe-9faa-4088-99c8-84ccb73daad2","Type":"ContainerStarted","Data":"b66ed08a2a59c66f9f0cab2693242bf28d7148ae5cb442d5c8f8db1a1b88ed7c"}
May 25 14:00:27.964818 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.964781    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-84d47c6dcc-6lqfw"]
May 25 14:00:27.985167 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.985129    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84d47c6dcc-6lqfw"]
May 25 14:00:27.985323 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.985192    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84d47c6dcc-6lqfw"
May 25 14:00:27.987892 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.987866    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-npnx8\""
May 25 14:00:27.988105 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.988092    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
May 25 14:00:27.988326 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.988310    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
May 25 14:00:27.988484 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.988471    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
May 25 14:00:27.988678 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.988665    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
May 25 14:00:27.988835 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.988823    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
May 25 14:00:27.989045 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.989022    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
May 25 14:00:27.989167 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:27.989101    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
May 25 14:00:28.066788 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:28.066736    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-console-oauth-config\") pod \"console-84d47c6dcc-6lqfw\" (UID: \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\") " pod="openshift-console/console-84d47c6dcc-6lqfw"
May 25 14:00:28.067006 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:28.066825    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-oauth-serving-cert\") pod \"console-84d47c6dcc-6lqfw\" (UID: \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\") " pod="openshift-console/console-84d47c6dcc-6lqfw"
May 25 14:00:28.067006 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:28.066863    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-console-serving-cert\") pod \"console-84d47c6dcc-6lqfw\" (UID: \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\") " pod="openshift-console/console-84d47c6dcc-6lqfw"
May 25 14:00:28.067141 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:28.066996    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-service-ca\") pod \"console-84d47c6dcc-6lqfw\" (UID: \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\") " pod="openshift-console/console-84d47c6dcc-6lqfw"
May 25 14:00:28.067141 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:28.067043    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-console-config\") pod \"console-84d47c6dcc-6lqfw\" (UID: \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\") " pod="openshift-console/console-84d47c6dcc-6lqfw"
May 25 14:00:28.067331 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:28.067303    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8ct5\" (UniqueName: \"kubernetes.io/projected/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-kube-api-access-z8ct5\") pod \"console-84d47c6dcc-6lqfw\" (UID: \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\") " pod="openshift-console/console-84d47c6dcc-6lqfw"
May 25 14:00:28.168457 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:28.168369    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-console-oauth-config\") pod \"console-84d47c6dcc-6lqfw\" (UID: \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\") " pod="openshift-console/console-84d47c6dcc-6lqfw"
May 25 14:00:28.172267 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:28.169326    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-oauth-serving-cert\") pod \"console-84d47c6dcc-6lqfw\" (UID: \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\") " pod="openshift-console/console-84d47c6dcc-6lqfw"
May 25 14:00:28.172267 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:28.169791    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-oauth-serving-cert\") pod \"console-84d47c6dcc-6lqfw\" (UID: \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\") " pod="openshift-console/console-84d47c6dcc-6lqfw"
May 25 14:00:28.172267 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:28.169837    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-console-serving-cert\") pod \"console-84d47c6dcc-6lqfw\" (UID: \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\") " pod="openshift-console/console-84d47c6dcc-6lqfw"
May 25 14:00:28.172267 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:28.169893    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-service-ca\") pod \"console-84d47c6dcc-6lqfw\" (UID: \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\") " pod="openshift-console/console-84d47c6dcc-6lqfw"
May 25 14:00:28.172267 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:28.169918    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-console-config\") pod \"console-84d47c6dcc-6lqfw\" (UID: \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\") " pod="openshift-console/console-84d47c6dcc-6lqfw"
May 25 14:00:28.172267 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:28.169967    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z8ct5\" (UniqueName: \"kubernetes.io/projected/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-kube-api-access-z8ct5\") pod \"console-84d47c6dcc-6lqfw\" (UID: \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\") " pod="openshift-console/console-84d47c6dcc-6lqfw"
May 25 14:00:28.172768 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:28.172569    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-service-ca\") pod \"console-84d47c6dcc-6lqfw\" (UID: \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\") " pod="openshift-console/console-84d47c6dcc-6lqfw"
May 25 14:00:28.172881 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:28.172618    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-console-config\") pod \"console-84d47c6dcc-6lqfw\" (UID: \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\") " pod="openshift-console/console-84d47c6dcc-6lqfw"
May 25 14:00:28.173811 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:28.173760    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-console-oauth-config\") pod \"console-84d47c6dcc-6lqfw\" (UID: \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\") " pod="openshift-console/console-84d47c6dcc-6lqfw"
May 25 14:00:28.174435 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:28.174256    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-console-serving-cert\") pod \"console-84d47c6dcc-6lqfw\" (UID: \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\") " pod="openshift-console/console-84d47c6dcc-6lqfw"
May 25 14:00:28.178123 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:28.178096    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8ct5\" (UniqueName: \"kubernetes.io/projected/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-kube-api-access-z8ct5\") pod \"console-84d47c6dcc-6lqfw\" (UID: \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\") " pod="openshift-console/console-84d47c6dcc-6lqfw"
May 25 14:00:28.298563 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:28.298516    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84d47c6dcc-6lqfw"
May 25 14:00:28.821753 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:28.821700    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 14:00:28.824804 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:28.824774    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sdv6d\""
May 25 14:00:28.825083 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:28.825061    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 25 14:00:30.283615 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:30.283584    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84d47c6dcc-6lqfw"]
May 25 14:00:30.901040 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:00:30.900995    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb3f10fb_2b34_4f98_9f6d_24b94bd20306.slice/crio-a5a253001aa3707f3aa501c9a481f026616b307912c63a3d4409af9f1157fc39 WatchSource:0}: Error finding container a5a253001aa3707f3aa501c9a481f026616b307912c63a3d4409af9f1157fc39: Status 404 returned error can't find the container with id a5a253001aa3707f3aa501c9a481f026616b307912c63a3d4409af9f1157fc39
May 25 14:00:30.963695 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:30.963655    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84d47c6dcc-6lqfw" event={"ID":"bb3f10fb-2b34-4f98-9f6d-24b94bd20306","Type":"ContainerStarted","Data":"a5a253001aa3707f3aa501c9a481f026616b307912c63a3d4409af9f1157fc39"}
May 25 14:00:31.554039 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.553952    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b465c95c9-vrtf8"]
May 25 14:00:31.563120 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.563086    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:31.569127 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.569092    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b465c95c9-vrtf8"]
May 25 14:00:31.571947 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.571905    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
May 25 14:00:31.700946 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.700893    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68d63673-6d67-4c9c-9454-b2050a39c39d-trusted-ca-bundle\") pod \"console-7b465c95c9-vrtf8\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") " pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:31.700946 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.700952    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68d63673-6d67-4c9c-9454-b2050a39c39d-oauth-serving-cert\") pod \"console-7b465c95c9-vrtf8\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") " pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:31.701157 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.701000    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68d63673-6d67-4c9c-9454-b2050a39c39d-console-config\") pod \"console-7b465c95c9-vrtf8\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") " pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:31.701157 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.701073    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68d63673-6d67-4c9c-9454-b2050a39c39d-console-serving-cert\") pod \"console-7b465c95c9-vrtf8\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") " pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:31.701157 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.701105    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68d63673-6d67-4c9c-9454-b2050a39c39d-console-oauth-config\") pod \"console-7b465c95c9-vrtf8\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") " pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:31.701157 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.701139    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hk8j\" (UniqueName: \"kubernetes.io/projected/68d63673-6d67-4c9c-9454-b2050a39c39d-kube-api-access-8hk8j\") pod \"console-7b465c95c9-vrtf8\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") " pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:31.701290 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.701168    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68d63673-6d67-4c9c-9454-b2050a39c39d-service-ca\") pod \"console-7b465c95c9-vrtf8\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") " pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:31.802104 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.802061    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8hk8j\" (UniqueName: \"kubernetes.io/projected/68d63673-6d67-4c9c-9454-b2050a39c39d-kube-api-access-8hk8j\") pod \"console-7b465c95c9-vrtf8\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") " pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:31.802278 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.802120    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68d63673-6d67-4c9c-9454-b2050a39c39d-service-ca\") pod \"console-7b465c95c9-vrtf8\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") " pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:31.802278 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.802147    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68d63673-6d67-4c9c-9454-b2050a39c39d-trusted-ca-bundle\") pod \"console-7b465c95c9-vrtf8\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") " pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:31.802278 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.802164    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68d63673-6d67-4c9c-9454-b2050a39c39d-oauth-serving-cert\") pod \"console-7b465c95c9-vrtf8\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") " pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:31.802278 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.802181    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68d63673-6d67-4c9c-9454-b2050a39c39d-console-config\") pod \"console-7b465c95c9-vrtf8\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") " pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:31.802278 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.802221    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68d63673-6d67-4c9c-9454-b2050a39c39d-console-serving-cert\") pod \"console-7b465c95c9-vrtf8\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") " pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:31.802278 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.802280    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68d63673-6d67-4c9c-9454-b2050a39c39d-console-oauth-config\") pod \"console-7b465c95c9-vrtf8\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") " pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:31.803043 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.803006    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68d63673-6d67-4c9c-9454-b2050a39c39d-console-config\") pod \"console-7b465c95c9-vrtf8\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") " pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:31.803215 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.803088    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68d63673-6d67-4c9c-9454-b2050a39c39d-trusted-ca-bundle\") pod \"console-7b465c95c9-vrtf8\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") " pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:31.803215 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.803089    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68d63673-6d67-4c9c-9454-b2050a39c39d-service-ca\") pod \"console-7b465c95c9-vrtf8\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") " pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:31.803215 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.803138    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68d63673-6d67-4c9c-9454-b2050a39c39d-oauth-serving-cert\") pod \"console-7b465c95c9-vrtf8\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") " pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:31.805258 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.805209    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68d63673-6d67-4c9c-9454-b2050a39c39d-console-oauth-config\") pod \"console-7b465c95c9-vrtf8\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") " pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:31.805415 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.805398    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68d63673-6d67-4c9c-9454-b2050a39c39d-console-serving-cert\") pod \"console-7b465c95c9-vrtf8\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") " pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:31.813479 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.810110    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hk8j\" (UniqueName: \"kubernetes.io/projected/68d63673-6d67-4c9c-9454-b2050a39c39d-kube-api-access-8hk8j\") pod \"console-7b465c95c9-vrtf8\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") " pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:31.874905 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.874860    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:31.967680 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.967647    2571 generic.go:358] "Generic (PLEG): container finished" podID="dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f" containerID="e23581d66c2eb1946ddd8dcad32a0295fdfad796553d3c2e082a17b907a02e31" exitCode=0
May 25 14:00:31.967836 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:31.967704    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-msrg9" event={"ID":"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f","Type":"ContainerDied","Data":"e23581d66c2eb1946ddd8dcad32a0295fdfad796553d3c2e082a17b907a02e31"}
May 25 14:00:32.249427 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:32.249033    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b465c95c9-vrtf8"]
May 25 14:00:32.972590 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:32.972531    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sn7fl" event={"ID":"678dec39-f2e8-486d-9f09-56a72f3cdd85","Type":"ContainerStarted","Data":"57c2f398b871a1f8c9302f2b9ed94e28ac77ea8df1174330a396f9febca322aa"}
May 25 14:00:32.974614 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:32.974582    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b465c95c9-vrtf8" event={"ID":"68d63673-6d67-4c9c-9454-b2050a39c39d","Type":"ContainerStarted","Data":"4ddac8977ce104a38f2a40c86349cbf76ce1d338ee6a04a0e9eeb5fd6a0e91c8"}
May 25 14:00:32.977491 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:32.977454    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5kgzp" event={"ID":"24931dfe-9faa-4088-99c8-84ccb73daad2","Type":"ContainerStarted","Data":"c75c3cbca2a01c3bff643bae1ba555aa75756f7d30e70009620f38d2fd0303f1"}
May 25 14:00:32.982272 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:32.982231    2571 generic.go:358] "Generic (PLEG): container finished" podID="dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f" containerID="26f90531527a7223dbe9428d6fbe9ce49af4eadde805b892e323ffc4bf8fa010" exitCode=0
May 25 14:00:32.982395 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:32.982327    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-msrg9" event={"ID":"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f","Type":"ContainerDied","Data":"26f90531527a7223dbe9428d6fbe9ce49af4eadde805b892e323ffc4bf8fa010"}
May 25 14:00:32.986063 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:32.986034    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jqs89" event={"ID":"68592d9d-e82d-4b8f-9be9-572afeaf463c","Type":"ContainerStarted","Data":"8f39ddc52756e2779d3bf1835b3f2f488c641929235467b7f22ccabe50bf2193"}
May 25 14:00:32.986217 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:32.986071    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jqs89" event={"ID":"68592d9d-e82d-4b8f-9be9-572afeaf463c","Type":"ContainerStarted","Data":"087cd1b341b9a730756565af4f5560b1cabdc6787fb545576cd2be0a825f17a6"}
May 25 14:00:32.986375 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:32.986347    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jqs89"
May 25 14:00:33.013533 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.013484    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-sn7fl" podStartSLOduration=1.504926254 podStartE2EDuration="6.013470073s" podCreationTimestamp="2026-05-25 14:00:27 +0000 UTC" firstStartedPulling="2026-05-25 14:00:27.551142688 +0000 UTC m=+35.385998764" lastFinishedPulling="2026-05-25 14:00:32.059686499 +0000 UTC m=+39.894542583" observedRunningTime="2026-05-25 14:00:32.98998331 +0000 UTC m=+40.824839405" watchObservedRunningTime="2026-05-25 14:00:33.013470073 +0000 UTC m=+40.848326165"
May 25 14:00:33.032497 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.032237    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-jqs89" podStartSLOduration=1.5806624519999999 podStartE2EDuration="6.032214937s" podCreationTimestamp="2026-05-25 14:00:27 +0000 UTC" firstStartedPulling="2026-05-25 14:00:27.641527937 +0000 UTC m=+35.476384011" lastFinishedPulling="2026-05-25 14:00:32.09308041 +0000 UTC m=+39.927936496" observedRunningTime="2026-05-25 14:00:33.030909396 +0000 UTC m=+40.865765500" watchObservedRunningTime="2026-05-25 14:00:33.032214937 +0000 UTC m=+40.867071032"
May 25 14:00:33.533182 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.533142    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-rxqh2"]
May 25 14:00:33.535596 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.535563    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-9c959559f-rxqh2"
May 25 14:00:33.542289 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.541617    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
May 25 14:00:33.542289 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.541774    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 25 14:00:33.542289 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.542104    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
May 25 14:00:33.542289 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.542278    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 25 14:00:33.542581 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.542541    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 25 14:00:33.546901 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.546878    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-k8gfm\""
May 25 14:00:33.562029 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.561994    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dnnst"]
May 25 14:00:33.564579 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.564558    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.567844 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.567818    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 25 14:00:33.567993 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.567867    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 25 14:00:33.567993 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.567908    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-47fwk\""
May 25 14:00:33.568079 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.567824    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 25 14:00:33.576841 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.576782    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7"]
May 25 14:00:33.579088 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.579061    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-rxqh2"]
May 25 14:00:33.579229 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.579211    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7"
May 25 14:00:33.582137 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.581868    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
May 25 14:00:33.582137 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.581870    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
May 25 14:00:33.582137 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.581898    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-tpwfl\""
May 25 14:00:33.582137 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.582031    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
May 25 14:00:33.589380 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.589352    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7"]
May 25 14:00:33.719219 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.719101    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7e2be626-b053-44b8-b177-c43e4e451265-root\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.719219 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.719147    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7e2be626-b053-44b8-b177-c43e4e451265-node-exporter-tls\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.719219 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.719179    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e2be626-b053-44b8-b177-c43e4e451265-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.719503 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.719264    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/75bd5dfa-11be-44fe-b4da-b5c06ddee30a-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-nvqq7\" (UID: \"75bd5dfa-11be-44fe-b4da-b5c06ddee30a\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7"
May 25 14:00:33.719503 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.719331    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/77b90865-975f-46ae-bb1f-e7cf87aef32a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-rxqh2\" (UID: \"77b90865-975f-46ae-bb1f-e7cf87aef32a\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-rxqh2"
May 25 14:00:33.719503 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.719367    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7e2be626-b053-44b8-b177-c43e4e451265-node-exporter-textfile\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.719503 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.719406    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/75bd5dfa-11be-44fe-b4da-b5c06ddee30a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-nvqq7\" (UID: \"75bd5dfa-11be-44fe-b4da-b5c06ddee30a\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7"
May 25 14:00:33.719503 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.719475    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/75bd5dfa-11be-44fe-b4da-b5c06ddee30a-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-nvqq7\" (UID: \"75bd5dfa-11be-44fe-b4da-b5c06ddee30a\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7"
May 25 14:00:33.719503 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.719500    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pqv6\" (UniqueName: \"kubernetes.io/projected/77b90865-975f-46ae-bb1f-e7cf87aef32a-kube-api-access-6pqv6\") pod \"openshift-state-metrics-9c959559f-rxqh2\" (UID: \"77b90865-975f-46ae-bb1f-e7cf87aef32a\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-rxqh2"
May 25 14:00:33.719803 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.719526    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77b90865-975f-46ae-bb1f-e7cf87aef32a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-rxqh2\" (UID: \"77b90865-975f-46ae-bb1f-e7cf87aef32a\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-rxqh2"
May 25 14:00:33.719803 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.719543    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/75bd5dfa-11be-44fe-b4da-b5c06ddee30a-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-nvqq7\" (UID: \"75bd5dfa-11be-44fe-b4da-b5c06ddee30a\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7"
May 25 14:00:33.719803 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.719649    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7e2be626-b053-44b8-b177-c43e4e451265-node-exporter-wtmp\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.719803 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.719681    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e2be626-b053-44b8-b177-c43e4e451265-metrics-client-ca\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.719803 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.719709    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/75bd5dfa-11be-44fe-b4da-b5c06ddee30a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-nvqq7\" (UID: \"75bd5dfa-11be-44fe-b4da-b5c06ddee30a\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7"
May 25 14:00:33.719803 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.719731    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7e2be626-b053-44b8-b177-c43e4e451265-node-exporter-accelerators-collector-config\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.719803 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.719756    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j787z\" (UniqueName: \"kubernetes.io/projected/7e2be626-b053-44b8-b177-c43e4e451265-kube-api-access-j787z\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.719803 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.719793    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mldmt\" (UniqueName: \"kubernetes.io/projected/75bd5dfa-11be-44fe-b4da-b5c06ddee30a-kube-api-access-mldmt\") pod \"kube-state-metrics-54fd6bc5c4-nvqq7\" (UID: \"75bd5dfa-11be-44fe-b4da-b5c06ddee30a\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7"
May 25 14:00:33.720223 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.719820    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77b90865-975f-46ae-bb1f-e7cf87aef32a-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-rxqh2\" (UID: \"77b90865-975f-46ae-bb1f-e7cf87aef32a\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-rxqh2"
May 25 14:00:33.720223 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.719847    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e2be626-b053-44b8-b177-c43e4e451265-sys\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.821144 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.821051    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7e2be626-b053-44b8-b177-c43e4e451265-node-exporter-wtmp\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.821144 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.821096    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e2be626-b053-44b8-b177-c43e4e451265-metrics-client-ca\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.821144 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.821124    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/75bd5dfa-11be-44fe-b4da-b5c06ddee30a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-nvqq7\" (UID: \"75bd5dfa-11be-44fe-b4da-b5c06ddee30a\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7"
May 25 14:00:33.821144 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.821145    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7e2be626-b053-44b8-b177-c43e4e451265-node-exporter-accelerators-collector-config\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.821402 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.821161    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j787z\" (UniqueName: \"kubernetes.io/projected/7e2be626-b053-44b8-b177-c43e4e451265-kube-api-access-j787z\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.821402 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.821181    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mldmt\" (UniqueName: \"kubernetes.io/projected/75bd5dfa-11be-44fe-b4da-b5c06ddee30a-kube-api-access-mldmt\") pod \"kube-state-metrics-54fd6bc5c4-nvqq7\" (UID: \"75bd5dfa-11be-44fe-b4da-b5c06ddee30a\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7"
May 25 14:00:33.821402 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.821206    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77b90865-975f-46ae-bb1f-e7cf87aef32a-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-rxqh2\" (UID: \"77b90865-975f-46ae-bb1f-e7cf87aef32a\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-rxqh2"
May 25 14:00:33.821402 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.821226    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e2be626-b053-44b8-b177-c43e4e451265-sys\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.821402 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.821249    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7e2be626-b053-44b8-b177-c43e4e451265-root\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.821402 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.821272    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7e2be626-b053-44b8-b177-c43e4e451265-node-exporter-tls\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.821402 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.821301    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e2be626-b053-44b8-b177-c43e4e451265-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.821402 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.821324    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/75bd5dfa-11be-44fe-b4da-b5c06ddee30a-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-nvqq7\" (UID: \"75bd5dfa-11be-44fe-b4da-b5c06ddee30a\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7"
May 25 14:00:33.821402 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.821343    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/77b90865-975f-46ae-bb1f-e7cf87aef32a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-rxqh2\" (UID: \"77b90865-975f-46ae-bb1f-e7cf87aef32a\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-rxqh2"
May 25 14:00:33.821402 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.821367    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7e2be626-b053-44b8-b177-c43e4e451265-node-exporter-textfile\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.821402 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.821384    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/75bd5dfa-11be-44fe-b4da-b5c06ddee30a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-nvqq7\" (UID: \"75bd5dfa-11be-44fe-b4da-b5c06ddee30a\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7"
May 25 14:00:33.821725 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.821413    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/75bd5dfa-11be-44fe-b4da-b5c06ddee30a-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-nvqq7\" (UID: \"75bd5dfa-11be-44fe-b4da-b5c06ddee30a\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7"
May 25 14:00:33.821725 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.821444    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6pqv6\" (UniqueName: \"kubernetes.io/projected/77b90865-975f-46ae-bb1f-e7cf87aef32a-kube-api-access-6pqv6\") pod \"openshift-state-metrics-9c959559f-rxqh2\" (UID: \"77b90865-975f-46ae-bb1f-e7cf87aef32a\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-rxqh2"
May 25 14:00:33.821725 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.821487    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77b90865-975f-46ae-bb1f-e7cf87aef32a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-rxqh2\" (UID: \"77b90865-975f-46ae-bb1f-e7cf87aef32a\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-rxqh2"
May 25 14:00:33.821725 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.821504    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/75bd5dfa-11be-44fe-b4da-b5c06ddee30a-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-nvqq7\" (UID: \"75bd5dfa-11be-44fe-b4da-b5c06ddee30a\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7"
May 25 14:00:33.821909 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.821870    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/75bd5dfa-11be-44fe-b4da-b5c06ddee30a-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-nvqq7\" (UID: \"75bd5dfa-11be-44fe-b4da-b5c06ddee30a\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7"
May 25 14:00:33.822015 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.821950    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7e2be626-b053-44b8-b177-c43e4e451265-root\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.822595 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.822568    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7e2be626-b053-44b8-b177-c43e4e451265-node-exporter-accelerators-collector-config\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.822805 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.822787    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7e2be626-b053-44b8-b177-c43e4e451265-node-exporter-wtmp\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.823098 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.823077    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7e2be626-b053-44b8-b177-c43e4e451265-node-exporter-textfile\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.823181 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.823158    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77b90865-975f-46ae-bb1f-e7cf87aef32a-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-rxqh2\" (UID: \"77b90865-975f-46ae-bb1f-e7cf87aef32a\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-rxqh2"
May 25 14:00:33.823256 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.823228    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e2be626-b053-44b8-b177-c43e4e451265-sys\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.824623 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.823680    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e2be626-b053-44b8-b177-c43e4e451265-metrics-client-ca\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.824623 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:33.823794    2571 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
May 25 14:00:33.824623 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:33.823854    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77b90865-975f-46ae-bb1f-e7cf87aef32a-openshift-state-metrics-tls podName:77b90865-975f-46ae-bb1f-e7cf87aef32a nodeName:}" failed. No retries permitted until 2026-05-25 14:00:34.32383383 +0000 UTC m=+42.158689901 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/77b90865-975f-46ae-bb1f-e7cf87aef32a-openshift-state-metrics-tls") pod "openshift-state-metrics-9c959559f-rxqh2" (UID: "77b90865-975f-46ae-bb1f-e7cf87aef32a") : secret "openshift-state-metrics-tls" not found
May 25 14:00:33.824623 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.824056    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/75bd5dfa-11be-44fe-b4da-b5c06ddee30a-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-nvqq7\" (UID: \"75bd5dfa-11be-44fe-b4da-b5c06ddee30a\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7"
May 25 14:00:33.824895 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.824648    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/75bd5dfa-11be-44fe-b4da-b5c06ddee30a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-nvqq7\" (UID: \"75bd5dfa-11be-44fe-b4da-b5c06ddee30a\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7"
May 25 14:00:33.829211 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.828837    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e2be626-b053-44b8-b177-c43e4e451265-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.829211 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.829075    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/75bd5dfa-11be-44fe-b4da-b5c06ddee30a-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-nvqq7\" (UID: \"75bd5dfa-11be-44fe-b4da-b5c06ddee30a\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7"
May 25 14:00:33.831066 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.831042    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/75bd5dfa-11be-44fe-b4da-b5c06ddee30a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-nvqq7\" (UID: \"75bd5dfa-11be-44fe-b4da-b5c06ddee30a\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7"
May 25 14:00:33.831544 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.831522    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j787z\" (UniqueName: \"kubernetes.io/projected/7e2be626-b053-44b8-b177-c43e4e451265-kube-api-access-j787z\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.831686 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.831610    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mldmt\" (UniqueName: \"kubernetes.io/projected/75bd5dfa-11be-44fe-b4da-b5c06ddee30a-kube-api-access-mldmt\") pod \"kube-state-metrics-54fd6bc5c4-nvqq7\" (UID: \"75bd5dfa-11be-44fe-b4da-b5c06ddee30a\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7"
May 25 14:00:33.831997 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.831947    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7e2be626-b053-44b8-b177-c43e4e451265-node-exporter-tls\") pod \"node-exporter-dnnst\" (UID: \"7e2be626-b053-44b8-b177-c43e4e451265\") " pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.832148 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.832126    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77b90865-975f-46ae-bb1f-e7cf87aef32a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-rxqh2\" (UID: \"77b90865-975f-46ae-bb1f-e7cf87aef32a\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-rxqh2"
May 25 14:00:33.833146 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.833126    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pqv6\" (UniqueName: \"kubernetes.io/projected/77b90865-975f-46ae-bb1f-e7cf87aef32a-kube-api-access-6pqv6\") pod \"openshift-state-metrics-9c959559f-rxqh2\" (UID: \"77b90865-975f-46ae-bb1f-e7cf87aef32a\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-rxqh2"
May 25 14:00:33.878573 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.878537    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dnnst"
May 25 14:00:33.893130 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.893091    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7"
May 25 14:00:33.991795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:33.991751    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-msrg9" event={"ID":"dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f","Type":"ContainerStarted","Data":"6897c4462cd2abb0b3518311663d47c43a672d8f7f3297cb227a73aac57fd075"}
May 25 14:00:34.012387 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.012323    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-msrg9" podStartSLOduration=5.6914539600000005 podStartE2EDuration="41.01230535s" podCreationTimestamp="2026-05-25 13:59:53 +0000 UTC" firstStartedPulling="2026-05-25 13:59:55.619321373 +0000 UTC m=+3.454177444" lastFinishedPulling="2026-05-25 14:00:30.940172754 +0000 UTC m=+38.775028834" observedRunningTime="2026-05-25 14:00:34.011573652 +0000 UTC m=+41.846429746" watchObservedRunningTime="2026-05-25 14:00:34.01230535 +0000 UTC m=+41.847161449"
May 25 14:00:34.326444 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.326405    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/77b90865-975f-46ae-bb1f-e7cf87aef32a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-rxqh2\" (UID: \"77b90865-975f-46ae-bb1f-e7cf87aef32a\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-rxqh2"
May 25 14:00:34.329342 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.329314    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/77b90865-975f-46ae-bb1f-e7cf87aef32a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-rxqh2\" (UID: \"77b90865-975f-46ae-bb1f-e7cf87aef32a\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-rxqh2"
May 25 14:00:34.448465 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.448429    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-9c959559f-rxqh2"
May 25 14:00:34.460821 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.460783    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 25 14:00:34.463553 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.463530    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.465602 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.465580    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
May 25 14:00:34.465788 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.465583    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
May 25 14:00:34.465963 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.465859    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
May 25 14:00:34.466057 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.465862    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
May 25 14:00:34.466275 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.466255    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
May 25 14:00:34.466381 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.466286    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
May 25 14:00:34.466381 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.466344    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
May 25 14:00:34.466381 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.466355    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
May 25 14:00:34.466544 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.466438    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
May 25 14:00:34.466650 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.466632    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-4zn45\""
May 25 14:00:34.477596 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.477563    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 25 14:00:34.528942 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.528901    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pffbv\" (UniqueName: \"kubernetes.io/projected/e6b44271-ffa5-4b01-b294-9e962e15927c-kube-api-access-pffbv\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.529127 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.528979    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.529127 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.529006    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6b44271-ffa5-4b01-b294-9e962e15927c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.529127 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.529040    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-config-volume\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.529127 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.529064    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e6b44271-ffa5-4b01-b294-9e962e15927c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.529127 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.529124    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.529345 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.529175    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.529345 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.529217    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6b44271-ffa5-4b01-b294-9e962e15927c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.529345 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.529242    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e6b44271-ffa5-4b01-b294-9e962e15927c-config-out\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.529345 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.529269    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-web-config\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.529345 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.529294    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.529345 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.529324    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.529623 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.529369    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e6b44271-ffa5-4b01-b294-9e962e15927c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.629833 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.629744    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.629833 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.629807    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.630090 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.629849    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6b44271-ffa5-4b01-b294-9e962e15927c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.630090 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.629874    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e6b44271-ffa5-4b01-b294-9e962e15927c-config-out\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.630090 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.629997    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-web-config\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.630090 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.630043    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.630090 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.630080    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.630325 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.630126    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e6b44271-ffa5-4b01-b294-9e962e15927c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.630325 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.630162    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pffbv\" (UniqueName: \"kubernetes.io/projected/e6b44271-ffa5-4b01-b294-9e962e15927c-kube-api-access-pffbv\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.630325 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.630195    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.630325 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.630220    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6b44271-ffa5-4b01-b294-9e962e15927c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.630325 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.630258    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-config-volume\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.630325 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.630288    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e6b44271-ffa5-4b01-b294-9e962e15927c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.630927 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.630614    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e6b44271-ffa5-4b01-b294-9e962e15927c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.630927 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:34.630833    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e6b44271-ffa5-4b01-b294-9e962e15927c-alertmanager-trusted-ca-bundle podName:e6b44271-ffa5-4b01-b294-9e962e15927c nodeName:}" failed. No retries permitted until 2026-05-25 14:00:35.130808418 +0000 UTC m=+42.965664503 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/e6b44271-ffa5-4b01-b294-9e962e15927c-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "e6b44271-ffa5-4b01-b294-9e962e15927c") : configmap references non-existent config key: ca-bundle.crt
May 25 14:00:34.630927 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.630870    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6b44271-ffa5-4b01-b294-9e962e15927c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.631381 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:34.631206    2571 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
May 25 14:00:34.631381 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:34.631258    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-main-tls podName:e6b44271-ffa5-4b01-b294-9e962e15927c nodeName:}" failed. No retries permitted until 2026-05-25 14:00:35.131239782 +0000 UTC m=+42.966095887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "e6b44271-ffa5-4b01-b294-9e962e15927c") : secret "alertmanager-main-tls" not found
May 25 14:00:34.633279 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.633252    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.633692 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.633667    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-web-config\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.634715 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.634283    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e6b44271-ffa5-4b01-b294-9e962e15927c-config-out\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.634715 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.634314    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-config-volume\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.634715 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.634653    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.634715 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.634660    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e6b44271-ffa5-4b01-b294-9e962e15927c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.634999 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.634848    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.636065 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.636041    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:34.638913 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:34.638889    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pffbv\" (UniqueName: \"kubernetes.io/projected/e6b44271-ffa5-4b01-b294-9e962e15927c-kube-api-access-pffbv\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:35.095775 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:00:35.095725    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e2be626_b053_44b8_b177_c43e4e451265.slice/crio-23dc9df350fca2abc2359cccba99efe4702cdc0f46d996f45fac6614279d146c WatchSource:0}: Error finding container 23dc9df350fca2abc2359cccba99efe4702cdc0f46d996f45fac6614279d146c: Status 404 returned error can't find the container with id 23dc9df350fca2abc2359cccba99efe4702cdc0f46d996f45fac6614279d146c
May 25 14:00:35.138159 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:35.137255    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:35.138159 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:35.137340    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6b44271-ffa5-4b01-b294-9e962e15927c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:35.138337 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:35.138312    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6b44271-ffa5-4b01-b294-9e962e15927c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:35.140759 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:35.140733    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:35.257377 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:35.257336    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7"]
May 25 14:00:35.261275 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:00:35.261246    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75bd5dfa_11be_44fe_b4da_b5c06ddee30a.slice/crio-fc150c8dc6a751bc74516be9d51533565c7239b916b72c00b5f80bbb951ef798 WatchSource:0}: Error finding container fc150c8dc6a751bc74516be9d51533565c7239b916b72c00b5f80bbb951ef798: Status 404 returned error can't find the container with id fc150c8dc6a751bc74516be9d51533565c7239b916b72c00b5f80bbb951ef798
May 25 14:00:35.274950 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:35.274882    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-rxqh2"]
May 25 14:00:35.278959 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:00:35.278649    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77b90865_975f_46ae_bb1f_e7cf87aef32a.slice/crio-d6c9be2baeaf4bb8074605525f4d8c549a44194751c86179c1e715914f744bc4 WatchSource:0}: Error finding container d6c9be2baeaf4bb8074605525f4d8c549a44194751c86179c1e715914f744bc4: Status 404 returned error can't find the container with id d6c9be2baeaf4bb8074605525f4d8c549a44194751c86179c1e715914f744bc4
May 25 14:00:35.375244 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:35.374791    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 25 14:00:35.524849 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:35.524812    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 25 14:00:35.529249 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:00:35.529222    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b44271_ffa5_4b01_b294_9e962e15927c.slice/crio-ae7c26fa1d4012ce371127553acb294d3d164e8d7f3e2f5ff595f6261e27d699 WatchSource:0}: Error finding container ae7c26fa1d4012ce371127553acb294d3d164e8d7f3e2f5ff595f6261e27d699: Status 404 returned error can't find the container with id ae7c26fa1d4012ce371127553acb294d3d164e8d7f3e2f5ff595f6261e27d699
May 25 14:00:36.000267 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.000225    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5kgzp" event={"ID":"24931dfe-9faa-4088-99c8-84ccb73daad2","Type":"ContainerStarted","Data":"773b8437d45cc39b1f7efe85237b636206ea00d7a9b1dc263994a062903169d6"}
May 25 14:00:36.001706 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.001653    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dnnst" event={"ID":"7e2be626-b053-44b8-b177-c43e4e451265","Type":"ContainerStarted","Data":"23dc9df350fca2abc2359cccba99efe4702cdc0f46d996f45fac6614279d146c"}
May 25 14:00:36.004065 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.002879    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e6b44271-ffa5-4b01-b294-9e962e15927c","Type":"ContainerStarted","Data":"ae7c26fa1d4012ce371127553acb294d3d164e8d7f3e2f5ff595f6261e27d699"}
May 25 14:00:36.005110 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.005030    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7" event={"ID":"75bd5dfa-11be-44fe-b4da-b5c06ddee30a","Type":"ContainerStarted","Data":"fc150c8dc6a751bc74516be9d51533565c7239b916b72c00b5f80bbb951ef798"}
May 25 14:00:36.006967 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.006919    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84d47c6dcc-6lqfw" event={"ID":"bb3f10fb-2b34-4f98-9f6d-24b94bd20306","Type":"ContainerStarted","Data":"8cecd418c55b19964b09ddeafe68ee2b2810670e70095f6f108f08d9f9361f25"}
May 25 14:00:36.010240 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.010213    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b465c95c9-vrtf8" event={"ID":"68d63673-6d67-4c9c-9454-b2050a39c39d","Type":"ContainerStarted","Data":"e902793d72c17769a822de5c457b67d551ff75f4221a7f44fcc1b712bdf7779b"}
May 25 14:00:36.013075 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.013009    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-rxqh2" event={"ID":"77b90865-975f-46ae-bb1f-e7cf87aef32a","Type":"ContainerStarted","Data":"301dd7680a91205050ca49df8da079a3141cfad9629b184eb85791be375d5c1c"}
May 25 14:00:36.013075 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.013042    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-rxqh2" event={"ID":"77b90865-975f-46ae-bb1f-e7cf87aef32a","Type":"ContainerStarted","Data":"28762c23aa76a37dc4f5aa56c2036ae305f9f7abaf700807b2f009a3b3837dee"}
May 25 14:00:36.013075 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.013056    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-rxqh2" event={"ID":"77b90865-975f-46ae-bb1f-e7cf87aef32a","Type":"ContainerStarted","Data":"d6c9be2baeaf4bb8074605525f4d8c549a44194751c86179c1e715914f744bc4"}
May 25 14:00:36.037993 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.037676    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-84d47c6dcc-6lqfw" podStartSLOduration=4.85020793 podStartE2EDuration="9.037655962s" podCreationTimestamp="2026-05-25 14:00:27 +0000 UTC" firstStartedPulling="2026-05-25 14:00:30.910828946 +0000 UTC m=+38.745685019" lastFinishedPulling="2026-05-25 14:00:35.098276969 +0000 UTC m=+42.933133051" observedRunningTime="2026-05-25 14:00:36.036613337 +0000 UTC m=+43.871469430" watchObservedRunningTime="2026-05-25 14:00:36.037655962 +0000 UTC m=+43.872512056"
May 25 14:00:36.038177 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.038126    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5kgzp" podStartSLOduration=2.620143592 podStartE2EDuration="10.038115035s" podCreationTimestamp="2026-05-25 14:00:26 +0000 UTC" firstStartedPulling="2026-05-25 14:00:27.679089209 +0000 UTC m=+35.513945281" lastFinishedPulling="2026-05-25 14:00:35.097060648 +0000 UTC m=+42.931916724" observedRunningTime="2026-05-25 14:00:36.018694924 +0000 UTC m=+43.853551018" watchObservedRunningTime="2026-05-25 14:00:36.038115035 +0000 UTC m=+43.872971130"
May 25 14:00:36.052804 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.052748    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-7b465c95c9-vrtf8" podStartSLOduration=2.208762263 podStartE2EDuration="5.052732663s" podCreationTimestamp="2026-05-25 14:00:31 +0000 UTC" firstStartedPulling="2026-05-25 14:00:32.262100972 +0000 UTC m=+40.096957054" lastFinishedPulling="2026-05-25 14:00:35.106071382 +0000 UTC m=+42.940927454" observedRunningTime="2026-05-25 14:00:36.052618995 +0000 UTC m=+43.887475101" watchObservedRunningTime="2026-05-25 14:00:36.052732663 +0000 UTC m=+43.887588756"
May 25 14:00:36.480147 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.480106    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5f69c45485-shw4q"]
May 25 14:00:36.483197 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.483169    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:36.485261 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.485232    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
May 25 14:00:36.485395 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.485312    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-j4bpl\""
May 25 14:00:36.485753 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.485528    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
May 25 14:00:36.485753 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.485575    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-917053cb1mhum\""
May 25 14:00:36.485753 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.485610    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
May 25 14:00:36.485753 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.485658    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
May 25 14:00:36.485753 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.485658    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
May 25 14:00:36.496221 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.495428    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5f69c45485-shw4q"]
May 25 14:00:36.551925 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.551889    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/95f37b20-7603-4a8a-9e85-72c406d458fa-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5f69c45485-shw4q\" (UID: \"95f37b20-7603-4a8a-9e85-72c406d458fa\") " pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:36.552121 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.551969    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/95f37b20-7603-4a8a-9e85-72c406d458fa-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5f69c45485-shw4q\" (UID: \"95f37b20-7603-4a8a-9e85-72c406d458fa\") " pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:36.552121 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.552019    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95f37b20-7603-4a8a-9e85-72c406d458fa-metrics-client-ca\") pod \"thanos-querier-5f69c45485-shw4q\" (UID: \"95f37b20-7603-4a8a-9e85-72c406d458fa\") " pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:36.552211 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.552142    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/95f37b20-7603-4a8a-9e85-72c406d458fa-secret-grpc-tls\") pod \"thanos-querier-5f69c45485-shw4q\" (UID: \"95f37b20-7603-4a8a-9e85-72c406d458fa\") " pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:36.552211 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.552197    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/95f37b20-7603-4a8a-9e85-72c406d458fa-secret-thanos-querier-tls\") pod \"thanos-querier-5f69c45485-shw4q\" (UID: \"95f37b20-7603-4a8a-9e85-72c406d458fa\") " pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:36.552308 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.552233    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/95f37b20-7603-4a8a-9e85-72c406d458fa-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5f69c45485-shw4q\" (UID: \"95f37b20-7603-4a8a-9e85-72c406d458fa\") " pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:36.552308 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.552256    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/95f37b20-7603-4a8a-9e85-72c406d458fa-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5f69c45485-shw4q\" (UID: \"95f37b20-7603-4a8a-9e85-72c406d458fa\") " pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:36.552308 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.552287    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp6bb\" (UniqueName: \"kubernetes.io/projected/95f37b20-7603-4a8a-9e85-72c406d458fa-kube-api-access-dp6bb\") pod \"thanos-querier-5f69c45485-shw4q\" (UID: \"95f37b20-7603-4a8a-9e85-72c406d458fa\") " pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:36.653544 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.653503    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/95f37b20-7603-4a8a-9e85-72c406d458fa-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5f69c45485-shw4q\" (UID: \"95f37b20-7603-4a8a-9e85-72c406d458fa\") " pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:36.653720 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.653654    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95f37b20-7603-4a8a-9e85-72c406d458fa-metrics-client-ca\") pod \"thanos-querier-5f69c45485-shw4q\" (UID: \"95f37b20-7603-4a8a-9e85-72c406d458fa\") " pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:36.653782 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.653723    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/95f37b20-7603-4a8a-9e85-72c406d458fa-secret-grpc-tls\") pod \"thanos-querier-5f69c45485-shw4q\" (UID: \"95f37b20-7603-4a8a-9e85-72c406d458fa\") " pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:36.653782 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.653759    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/95f37b20-7603-4a8a-9e85-72c406d458fa-secret-thanos-querier-tls\") pod \"thanos-querier-5f69c45485-shw4q\" (UID: \"95f37b20-7603-4a8a-9e85-72c406d458fa\") " pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:36.653881 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.653792    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/95f37b20-7603-4a8a-9e85-72c406d458fa-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5f69c45485-shw4q\" (UID: \"95f37b20-7603-4a8a-9e85-72c406d458fa\") " pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:36.653881 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.653826    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/95f37b20-7603-4a8a-9e85-72c406d458fa-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5f69c45485-shw4q\" (UID: \"95f37b20-7603-4a8a-9e85-72c406d458fa\") " pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:36.653881 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.653867    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dp6bb\" (UniqueName: \"kubernetes.io/projected/95f37b20-7603-4a8a-9e85-72c406d458fa-kube-api-access-dp6bb\") pod \"thanos-querier-5f69c45485-shw4q\" (UID: \"95f37b20-7603-4a8a-9e85-72c406d458fa\") " pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:36.654130 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.653911    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/95f37b20-7603-4a8a-9e85-72c406d458fa-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5f69c45485-shw4q\" (UID: \"95f37b20-7603-4a8a-9e85-72c406d458fa\") " pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:36.654501 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.654439    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95f37b20-7603-4a8a-9e85-72c406d458fa-metrics-client-ca\") pod \"thanos-querier-5f69c45485-shw4q\" (UID: \"95f37b20-7603-4a8a-9e85-72c406d458fa\") " pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:36.656703 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.656640    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/95f37b20-7603-4a8a-9e85-72c406d458fa-secret-grpc-tls\") pod \"thanos-querier-5f69c45485-shw4q\" (UID: \"95f37b20-7603-4a8a-9e85-72c406d458fa\") " pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:36.656865 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.656836    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/95f37b20-7603-4a8a-9e85-72c406d458fa-secret-thanos-querier-tls\") pod \"thanos-querier-5f69c45485-shw4q\" (UID: \"95f37b20-7603-4a8a-9e85-72c406d458fa\") " pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:36.657019 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.656997    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/95f37b20-7603-4a8a-9e85-72c406d458fa-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5f69c45485-shw4q\" (UID: \"95f37b20-7603-4a8a-9e85-72c406d458fa\") " pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:36.657019 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.657008    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/95f37b20-7603-4a8a-9e85-72c406d458fa-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5f69c45485-shw4q\" (UID: \"95f37b20-7603-4a8a-9e85-72c406d458fa\") " pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:36.657162 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.657100    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/95f37b20-7603-4a8a-9e85-72c406d458fa-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5f69c45485-shw4q\" (UID: \"95f37b20-7603-4a8a-9e85-72c406d458fa\") " pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:36.657338 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.657320    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/95f37b20-7603-4a8a-9e85-72c406d458fa-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5f69c45485-shw4q\" (UID: \"95f37b20-7603-4a8a-9e85-72c406d458fa\") " pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:36.661981 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.661961    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp6bb\" (UniqueName: \"kubernetes.io/projected/95f37b20-7603-4a8a-9e85-72c406d458fa-kube-api-access-dp6bb\") pod \"thanos-querier-5f69c45485-shw4q\" (UID: \"95f37b20-7603-4a8a-9e85-72c406d458fa\") " pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:36.796330 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:36.796233    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:37.094481 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.094417    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5f69c45485-shw4q"]
May 25 14:00:37.797434 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.797395    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-65898b7c8b-68bw5"]
May 25 14:00:37.799911 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.799889    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:00:37.801994 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.801969    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-lpth4\""
May 25 14:00:37.803230 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.802439    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
May 25 14:00:37.803230 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.802674    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
May 25 14:00:37.803230 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.802995    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
May 25 14:00:37.803230 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.803037    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
May 25 14:00:37.803230 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.803113    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-2m2a2u4srf69d\""
May 25 14:00:37.811620 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.811569    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-65898b7c8b-68bw5"]
May 25 14:00:37.866691 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.866649    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/95adfa22-3ac3-4d92-8c86-165fcb12a609-secret-metrics-server-client-certs\") pod \"metrics-server-65898b7c8b-68bw5\" (UID: \"95adfa22-3ac3-4d92-8c86-165fcb12a609\") " pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:00:37.866859 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.866700    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/95adfa22-3ac3-4d92-8c86-165fcb12a609-client-ca-bundle\") pod \"metrics-server-65898b7c8b-68bw5\" (UID: \"95adfa22-3ac3-4d92-8c86-165fcb12a609\") " pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:00:37.866859 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.866768    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/95adfa22-3ac3-4d92-8c86-165fcb12a609-audit-log\") pod \"metrics-server-65898b7c8b-68bw5\" (UID: \"95adfa22-3ac3-4d92-8c86-165fcb12a609\") " pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:00:37.866859 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.866800    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95adfa22-3ac3-4d92-8c86-165fcb12a609-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-65898b7c8b-68bw5\" (UID: \"95adfa22-3ac3-4d92-8c86-165fcb12a609\") " pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:00:37.866859 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.866828    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmkln\" (UniqueName: \"kubernetes.io/projected/95adfa22-3ac3-4d92-8c86-165fcb12a609-kube-api-access-nmkln\") pod \"metrics-server-65898b7c8b-68bw5\" (UID: \"95adfa22-3ac3-4d92-8c86-165fcb12a609\") " pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:00:37.867053 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.866980    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/95adfa22-3ac3-4d92-8c86-165fcb12a609-secret-metrics-server-tls\") pod \"metrics-server-65898b7c8b-68bw5\" (UID: \"95adfa22-3ac3-4d92-8c86-165fcb12a609\") " pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:00:37.867053 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.867013    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/95adfa22-3ac3-4d92-8c86-165fcb12a609-metrics-server-audit-profiles\") pod \"metrics-server-65898b7c8b-68bw5\" (UID: \"95adfa22-3ac3-4d92-8c86-165fcb12a609\") " pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:00:37.968147 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.968092    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/95adfa22-3ac3-4d92-8c86-165fcb12a609-audit-log\") pod \"metrics-server-65898b7c8b-68bw5\" (UID: \"95adfa22-3ac3-4d92-8c86-165fcb12a609\") " pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:00:37.968324 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.968163    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95adfa22-3ac3-4d92-8c86-165fcb12a609-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-65898b7c8b-68bw5\" (UID: \"95adfa22-3ac3-4d92-8c86-165fcb12a609\") " pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:00:37.968324 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.968194    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nmkln\" (UniqueName: \"kubernetes.io/projected/95adfa22-3ac3-4d92-8c86-165fcb12a609-kube-api-access-nmkln\") pod \"metrics-server-65898b7c8b-68bw5\" (UID: \"95adfa22-3ac3-4d92-8c86-165fcb12a609\") " pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:00:37.968324 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.968282    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/95adfa22-3ac3-4d92-8c86-165fcb12a609-secret-metrics-server-tls\") pod \"metrics-server-65898b7c8b-68bw5\" (UID: \"95adfa22-3ac3-4d92-8c86-165fcb12a609\") " pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:00:37.968324 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.968314    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/95adfa22-3ac3-4d92-8c86-165fcb12a609-metrics-server-audit-profiles\") pod \"metrics-server-65898b7c8b-68bw5\" (UID: \"95adfa22-3ac3-4d92-8c86-165fcb12a609\") " pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:00:37.968560 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.968536    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/95adfa22-3ac3-4d92-8c86-165fcb12a609-audit-log\") pod \"metrics-server-65898b7c8b-68bw5\" (UID: \"95adfa22-3ac3-4d92-8c86-165fcb12a609\") " pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:00:37.968620 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.968534    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/95adfa22-3ac3-4d92-8c86-165fcb12a609-secret-metrics-server-client-certs\") pod \"metrics-server-65898b7c8b-68bw5\" (UID: \"95adfa22-3ac3-4d92-8c86-165fcb12a609\") " pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:00:37.968620 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.968602    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/95adfa22-3ac3-4d92-8c86-165fcb12a609-client-ca-bundle\") pod \"metrics-server-65898b7c8b-68bw5\" (UID: \"95adfa22-3ac3-4d92-8c86-165fcb12a609\") " pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:00:37.969394 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.969270    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/95adfa22-3ac3-4d92-8c86-165fcb12a609-metrics-server-audit-profiles\") pod \"metrics-server-65898b7c8b-68bw5\" (UID: \"95adfa22-3ac3-4d92-8c86-165fcb12a609\") " pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:00:37.969501 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.969405    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95adfa22-3ac3-4d92-8c86-165fcb12a609-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-65898b7c8b-68bw5\" (UID: \"95adfa22-3ac3-4d92-8c86-165fcb12a609\") " pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:00:37.971755 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.971701    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/95adfa22-3ac3-4d92-8c86-165fcb12a609-secret-metrics-server-client-certs\") pod \"metrics-server-65898b7c8b-68bw5\" (UID: \"95adfa22-3ac3-4d92-8c86-165fcb12a609\") " pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:00:37.972090 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.972071    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/95adfa22-3ac3-4d92-8c86-165fcb12a609-client-ca-bundle\") pod \"metrics-server-65898b7c8b-68bw5\" (UID: \"95adfa22-3ac3-4d92-8c86-165fcb12a609\") " pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:00:37.972188 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.972160    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/95adfa22-3ac3-4d92-8c86-165fcb12a609-secret-metrics-server-tls\") pod \"metrics-server-65898b7c8b-68bw5\" (UID: \"95adfa22-3ac3-4d92-8c86-165fcb12a609\") " pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:00:37.976979 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:37.976951    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmkln\" (UniqueName: \"kubernetes.io/projected/95adfa22-3ac3-4d92-8c86-165fcb12a609-kube-api-access-nmkln\") pod \"metrics-server-65898b7c8b-68bw5\" (UID: \"95adfa22-3ac3-4d92-8c86-165fcb12a609\") " pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:00:38.035142 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.035099    2571 generic.go:358] "Generic (PLEG): container finished" podID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerID="d501924573d849186099b9933c4799ada759a9db9ff5846fecdb8ea3ffed1c5e" exitCode=0
May 25 14:00:38.035316 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.035184    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e6b44271-ffa5-4b01-b294-9e962e15927c","Type":"ContainerDied","Data":"d501924573d849186099b9933c4799ada759a9db9ff5846fecdb8ea3ffed1c5e"}
May 25 14:00:38.037818 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.037788    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7" event={"ID":"75bd5dfa-11be-44fe-b4da-b5c06ddee30a","Type":"ContainerStarted","Data":"51685780305a67e6a889789b508be6d5f5e1342b8e96d4280690a2f398d82b9c"}
May 25 14:00:38.037922 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.037829    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7" event={"ID":"75bd5dfa-11be-44fe-b4da-b5c06ddee30a","Type":"ContainerStarted","Data":"3ecb21171315a5712f75c5785ed7b53406e841b787fa4a59b0764135af3b4677"}
May 25 14:00:38.037922 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.037842    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7" event={"ID":"75bd5dfa-11be-44fe-b4da-b5c06ddee30a","Type":"ContainerStarted","Data":"214712a7f0a2e6d7ed86da2898ee68638af3521228119af5c63574c1622dc9ed"}
May 25 14:00:38.040397 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.040365    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-rxqh2" event={"ID":"77b90865-975f-46ae-bb1f-e7cf87aef32a","Type":"ContainerStarted","Data":"098a8e7c78e716e0cba066607698d23ce2ba6fea44a933618147af6e91c5f817"}
May 25 14:00:38.042092 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.042064    2571 generic.go:358] "Generic (PLEG): container finished" podID="7e2be626-b053-44b8-b177-c43e4e451265" containerID="4ce814565847dd576e0d3146d3daff2e03a5234b019e9f64b6f405db11537394" exitCode=0
May 25 14:00:38.042191 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.042140    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dnnst" event={"ID":"7e2be626-b053-44b8-b177-c43e4e451265","Type":"ContainerDied","Data":"4ce814565847dd576e0d3146d3daff2e03a5234b019e9f64b6f405db11537394"}
May 25 14:00:38.043849 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.043829    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q" event={"ID":"95f37b20-7603-4a8a-9e85-72c406d458fa","Type":"ContainerStarted","Data":"16cca9b9eda5e9970de0ee828fd93e121099b029a936bf4263aecd2f6f52e245"}
May 25 14:00:38.107590 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.107532    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-9c959559f-rxqh2" podStartSLOduration=3.583818533 podStartE2EDuration="5.107515976s" podCreationTimestamp="2026-05-25 14:00:33 +0000 UTC" firstStartedPulling="2026-05-25 14:00:35.400800339 +0000 UTC m=+43.235656414" lastFinishedPulling="2026-05-25 14:00:36.924497776 +0000 UTC m=+44.759353857" observedRunningTime="2026-05-25 14:00:38.105803715 +0000 UTC m=+45.940659809" watchObservedRunningTime="2026-05-25 14:00:38.107515976 +0000 UTC m=+45.942372068"
May 25 14:00:38.111195 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.111167    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:00:38.122683 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.122602    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-nvqq7" podStartSLOduration=3.461874797 podStartE2EDuration="5.12258251s" podCreationTimestamp="2026-05-25 14:00:33 +0000 UTC" firstStartedPulling="2026-05-25 14:00:35.263785757 +0000 UTC m=+43.098641833" lastFinishedPulling="2026-05-25 14:00:36.924493474 +0000 UTC m=+44.759349546" observedRunningTime="2026-05-25 14:00:38.122149894 +0000 UTC m=+45.957005988" watchObservedRunningTime="2026-05-25 14:00:38.12258251 +0000 UTC m=+45.957438603"
May 25 14:00:38.213539 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.212096    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-xlxsr"]
May 25 14:00:38.216951 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.216342    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-665c9c8584-xlxsr"
May 25 14:00:38.220905 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.220874    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
May 25 14:00:38.221374 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.221340    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-l9gm6\""
May 25 14:00:38.225050 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.225018    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-xlxsr"]
May 25 14:00:38.270454 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.270390    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-65898b7c8b-68bw5"]
May 25 14:00:38.271458 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.271431    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5521ae8-eced-4397-8e19-ae7b75b147e5-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-xlxsr\" (UID: \"d5521ae8-eced-4397-8e19-ae7b75b147e5\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-xlxsr"
May 25 14:00:38.274386 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:00:38.274353    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95adfa22_3ac3_4d92_8c86_165fcb12a609.slice/crio-b8dfd368ced6af0bf2d6c0042bb7c730e568f56ea1330d828af5c1053acf54ae WatchSource:0}: Error finding container b8dfd368ced6af0bf2d6c0042bb7c730e568f56ea1330d828af5c1053acf54ae: Status 404 returned error can't find the container with id b8dfd368ced6af0bf2d6c0042bb7c730e568f56ea1330d828af5c1053acf54ae
May 25 14:00:38.299738 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.299700    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84d47c6dcc-6lqfw"
May 25 14:00:38.299738 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.299742    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84d47c6dcc-6lqfw"
May 25 14:00:38.305167 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.305091    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-84d47c6dcc-6lqfw"
May 25 14:00:38.364662 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.364624    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84d47c6dcc-6lqfw"]
May 25 14:00:38.372820 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.372787    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5521ae8-eced-4397-8e19-ae7b75b147e5-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-xlxsr\" (UID: \"d5521ae8-eced-4397-8e19-ae7b75b147e5\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-xlxsr"
May 25 14:00:38.373021 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:38.372970    2571 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
May 25 14:00:38.373072 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:38.373050    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5521ae8-eced-4397-8e19-ae7b75b147e5-monitoring-plugin-cert podName:d5521ae8-eced-4397-8e19-ae7b75b147e5 nodeName:}" failed. No retries permitted until 2026-05-25 14:00:38.873026724 +0000 UTC m=+46.707882804 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/d5521ae8-eced-4397-8e19-ae7b75b147e5-monitoring-plugin-cert") pod "monitoring-plugin-665c9c8584-xlxsr" (UID: "d5521ae8-eced-4397-8e19-ae7b75b147e5") : secret "monitoring-plugin-cert" not found
May 25 14:00:38.388475 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.388430    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-df8bdcb5b-2gg87"]
May 25 14:00:38.391472 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.391449    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:38.402706 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.402680    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-df8bdcb5b-2gg87"]
May 25 14:00:38.473852 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.473816    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/334a1409-7274-4632-9443-6e81f9e2b6e2-console-config\") pod \"console-df8bdcb5b-2gg87\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") " pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:38.474062 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.473891    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/334a1409-7274-4632-9443-6e81f9e2b6e2-console-oauth-config\") pod \"console-df8bdcb5b-2gg87\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") " pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:38.474062 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.474006    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/334a1409-7274-4632-9443-6e81f9e2b6e2-service-ca\") pod \"console-df8bdcb5b-2gg87\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") " pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:38.474062 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.474035    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/334a1409-7274-4632-9443-6e81f9e2b6e2-trusted-ca-bundle\") pod \"console-df8bdcb5b-2gg87\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") " pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:38.474203 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.474089    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/334a1409-7274-4632-9443-6e81f9e2b6e2-console-serving-cert\") pod \"console-df8bdcb5b-2gg87\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") " pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:38.474203 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.474121    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrtx5\" (UniqueName: \"kubernetes.io/projected/334a1409-7274-4632-9443-6e81f9e2b6e2-kube-api-access-wrtx5\") pod \"console-df8bdcb5b-2gg87\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") " pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:38.474279 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.474209    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/334a1409-7274-4632-9443-6e81f9e2b6e2-oauth-serving-cert\") pod \"console-df8bdcb5b-2gg87\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") " pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:38.575157 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.575069    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/334a1409-7274-4632-9443-6e81f9e2b6e2-console-serving-cert\") pod \"console-df8bdcb5b-2gg87\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") " pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:38.575157 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.575129    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wrtx5\" (UniqueName: \"kubernetes.io/projected/334a1409-7274-4632-9443-6e81f9e2b6e2-kube-api-access-wrtx5\") pod \"console-df8bdcb5b-2gg87\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") " pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:38.575370 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.575166    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/334a1409-7274-4632-9443-6e81f9e2b6e2-oauth-serving-cert\") pod \"console-df8bdcb5b-2gg87\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") " pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:38.575370 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.575200    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/334a1409-7274-4632-9443-6e81f9e2b6e2-console-config\") pod \"console-df8bdcb5b-2gg87\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") " pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:38.575370 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.575245    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/334a1409-7274-4632-9443-6e81f9e2b6e2-console-oauth-config\") pod \"console-df8bdcb5b-2gg87\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") " pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:38.575370 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.575319    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/334a1409-7274-4632-9443-6e81f9e2b6e2-service-ca\") pod \"console-df8bdcb5b-2gg87\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") " pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:38.575370 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.575345    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/334a1409-7274-4632-9443-6e81f9e2b6e2-trusted-ca-bundle\") pod \"console-df8bdcb5b-2gg87\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") " pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:38.576528 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.576500    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/334a1409-7274-4632-9443-6e81f9e2b6e2-oauth-serving-cert\") pod \"console-df8bdcb5b-2gg87\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") " pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:38.576646 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.576537    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/334a1409-7274-4632-9443-6e81f9e2b6e2-trusted-ca-bundle\") pod \"console-df8bdcb5b-2gg87\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") " pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:38.576705 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.576654    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/334a1409-7274-4632-9443-6e81f9e2b6e2-service-ca\") pod \"console-df8bdcb5b-2gg87\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") " pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:38.577122 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.577091    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/334a1409-7274-4632-9443-6e81f9e2b6e2-console-config\") pod \"console-df8bdcb5b-2gg87\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") " pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:38.578143 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.578117    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/334a1409-7274-4632-9443-6e81f9e2b6e2-console-serving-cert\") pod \"console-df8bdcb5b-2gg87\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") " pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:38.578301 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.578281    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/334a1409-7274-4632-9443-6e81f9e2b6e2-console-oauth-config\") pod \"console-df8bdcb5b-2gg87\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") " pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:38.582555 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.582534    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrtx5\" (UniqueName: \"kubernetes.io/projected/334a1409-7274-4632-9443-6e81f9e2b6e2-kube-api-access-wrtx5\") pod \"console-df8bdcb5b-2gg87\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") " pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:38.703924 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.703883    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:38.878117 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.878081    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5521ae8-eced-4397-8e19-ae7b75b147e5-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-xlxsr\" (UID: \"d5521ae8-eced-4397-8e19-ae7b75b147e5\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-xlxsr"
May 25 14:00:38.880675 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.880645    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5521ae8-eced-4397-8e19-ae7b75b147e5-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-xlxsr\" (UID: \"d5521ae8-eced-4397-8e19-ae7b75b147e5\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-xlxsr"
May 25 14:00:38.937659 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:38.937626    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-df8bdcb5b-2gg87"]
May 25 14:00:38.940768 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:00:38.940738    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod334a1409_7274_4632_9443_6e81f9e2b6e2.slice/crio-eca646e4ec1438940affbb767d5742c7274419f904606421275d7a91bc586304 WatchSource:0}: Error finding container eca646e4ec1438940affbb767d5742c7274419f904606421275d7a91bc586304: Status 404 returned error can't find the container with id eca646e4ec1438940affbb767d5742c7274419f904606421275d7a91bc586304
May 25 14:00:39.048068 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.048029    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q" event={"ID":"95f37b20-7603-4a8a-9e85-72c406d458fa","Type":"ContainerStarted","Data":"f89e6e63cb2f2ea6a3fb7dfaf9396ef96c7cccb952936d8adfbc124eaa1f0c6f"}
May 25 14:00:39.049838 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.049196    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-df8bdcb5b-2gg87" event={"ID":"334a1409-7274-4632-9443-6e81f9e2b6e2","Type":"ContainerStarted","Data":"207bc0eb010d4280947e91231bd3141e92f4a2314e5b0bbd155e086f721e202e"}
May 25 14:00:39.049838 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.049220    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-df8bdcb5b-2gg87" event={"ID":"334a1409-7274-4632-9443-6e81f9e2b6e2","Type":"ContainerStarted","Data":"eca646e4ec1438940affbb767d5742c7274419f904606421275d7a91bc586304"}
May 25 14:00:39.051320 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.051188    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5" event={"ID":"95adfa22-3ac3-4d92-8c86-165fcb12a609","Type":"ContainerStarted","Data":"b8dfd368ced6af0bf2d6c0042bb7c730e568f56ea1330d828af5c1053acf54ae"}
May 25 14:00:39.053905 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.053859    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dnnst" event={"ID":"7e2be626-b053-44b8-b177-c43e4e451265","Type":"ContainerStarted","Data":"11bbd746e09da78812bd0aa57905a820a85ad0f921bd4efab33542c961a5e31c"}
May 25 14:00:39.054587 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.054050    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dnnst" event={"ID":"7e2be626-b053-44b8-b177-c43e4e451265","Type":"ContainerStarted","Data":"e51e9d5e8c34bf1aec174090885e4757b05a767642492fd6f0f51deee98780c1"}
May 25 14:00:39.059494 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.059464    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-84d47c6dcc-6lqfw"
May 25 14:00:39.067168 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.067106    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-df8bdcb5b-2gg87" podStartSLOduration=1.067087253 podStartE2EDuration="1.067087253s" podCreationTimestamp="2026-05-25 14:00:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 14:00:39.066323268 +0000 UTC m=+46.901179355" watchObservedRunningTime="2026-05-25 14:00:39.067087253 +0000 UTC m=+46.901943344"
May 25 14:00:39.104251 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.102494    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dnnst" podStartSLOduration=4.276483322 podStartE2EDuration="6.102476575s" podCreationTimestamp="2026-05-25 14:00:33 +0000 UTC" firstStartedPulling="2026-05-25 14:00:35.098501994 +0000 UTC m=+42.933358082" lastFinishedPulling="2026-05-25 14:00:36.92449525 +0000 UTC m=+44.759351335" observedRunningTime="2026-05-25 14:00:39.101504171 +0000 UTC m=+46.936360263" watchObservedRunningTime="2026-05-25 14:00:39.102476575 +0000 UTC m=+46.937332667"
May 25 14:00:39.133844 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.133817    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-665c9c8584-xlxsr"
May 25 14:00:39.316364 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.316308    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-xlxsr"]
May 25 14:00:39.324224 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:00:39.324191    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5521ae8_eced_4397_8e19_ae7b75b147e5.slice/crio-d20a8f2c8bff8358073d19fe447f8c15d100597d5a698021a543c23be895fb74 WatchSource:0}: Error finding container d20a8f2c8bff8358073d19fe447f8c15d100597d5a698021a543c23be895fb74: Status 404 returned error can't find the container with id d20a8f2c8bff8358073d19fe447f8c15d100597d5a698021a543c23be895fb74
May 25 14:00:39.594744 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.594706    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 25 14:00:39.607247 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.607218    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.615249 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.610400    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 25 14:00:39.615249 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.610637    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-br2gdh0mi2rfg\""
May 25 14:00:39.615249 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.610713    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
May 25 14:00:39.615249 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.610849    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
May 25 14:00:39.615249 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.611037    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
May 25 14:00:39.615249 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.611260    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
May 25 14:00:39.615249 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.611435    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
May 25 14:00:39.615249 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.611664    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
May 25 14:00:39.615249 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.611846    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
May 25 14:00:39.615249 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.611893    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-lp57l\""
May 25 14:00:39.615249 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.610637    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
May 25 14:00:39.615249 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.612108    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
May 25 14:00:39.615249 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.612287    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
May 25 14:00:39.615249 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.613069    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
May 25 14:00:39.615249 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.613128    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
May 25 14:00:39.688467 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.688428    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-config-out\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.688676 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.688499    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-config\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.688676 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.688528    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.688676 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.688557    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.688676 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.688584    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.688676 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.688659    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.688884 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.688697    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdxvd\" (UniqueName: \"kubernetes.io/projected/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-kube-api-access-fdxvd\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.688884 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.688724    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.688884 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.688799    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-web-config\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.688884 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.688839    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.688884 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.688863    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.688884 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.688885    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.689139 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.688906    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.689139 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.689014    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.689139 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.689058    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.689139 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.689084    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.689139 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.689118    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.689325 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.689150    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.790347 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.790309    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.790524 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.790366    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.790524 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.790400    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.790524 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.790421    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.790524 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.790461    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.790524 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.790487    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.790524 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.790513    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.790837 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.790539    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.790837 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.790577    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.790837 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.790633    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-config-out\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.790837 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.790692    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-config\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.790837 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.790722    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.790837 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.790754    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.790837 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.790784    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.790837 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.790826    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.791226 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.790853    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fdxvd\" (UniqueName: \"kubernetes.io/projected/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-kube-api-access-fdxvd\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.791758 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.791336    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.791758 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.791394    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-web-config\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.791957 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.791818    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.792898 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.792778    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.794558 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.794532    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.795407 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.795384    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.796351 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.796321    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.797004 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.796977    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.797107 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.797081    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.798492 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.797971    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.798492 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.798088    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.798492 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.798449    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.798991 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.798957    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-config-out\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.799492 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.799446    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.799617 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.799595    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.800156 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.800071    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdxvd\" (UniqueName: \"kubernetes.io/projected/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-kube-api-access-fdxvd\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.800533 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.800508    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.800655 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.800636    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-web-config\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.801743 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.801719    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.803364 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.803344    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-config\") pod \"prometheus-k8s-0\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:39.921715 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:39.921610    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:40.059056 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:40.059010    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q" event={"ID":"95f37b20-7603-4a8a-9e85-72c406d458fa","Type":"ContainerStarted","Data":"0df4021f2b4accae344de6e7ff40651a960c83c608bdd225f01bd8a9e5d5b793"}
May 25 14:00:40.059056 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:40.059063    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q" event={"ID":"95f37b20-7603-4a8a-9e85-72c406d458fa","Type":"ContainerStarted","Data":"ec7f030ab0a650e5cad328084f6fa4223eb00101ba5521797a3f0e5b31789a34"}
May 25 14:00:40.060317 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:40.060269    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-665c9c8584-xlxsr" event={"ID":"d5521ae8-eced-4397-8e19-ae7b75b147e5","Type":"ContainerStarted","Data":"d20a8f2c8bff8358073d19fe447f8c15d100597d5a698021a543c23be895fb74"}
May 25 14:00:40.259810 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:40.259728    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 25 14:00:40.430020 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:00:40.429979    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73a1b7d9_68a3_47a6_a3b2_1f3afb3bebaf.slice/crio-17c28cb8409f2713bcacc80667595728478df0db2d849208f311b9f45bea3d5d WatchSource:0}: Error finding container 17c28cb8409f2713bcacc80667595728478df0db2d849208f311b9f45bea3d5d: Status 404 returned error can't find the container with id 17c28cb8409f2713bcacc80667595728478df0db2d849208f311b9f45bea3d5d
May 25 14:00:41.064850 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:41.064814    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5" event={"ID":"95adfa22-3ac3-4d92-8c86-165fcb12a609","Type":"ContainerStarted","Data":"2a57ca0b7c579773d5a832ca8d2ff2fe820a99006992db96f4b60979d65bdbf1"}
May 25 14:00:41.066269 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:41.066242    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf","Type":"ContainerStarted","Data":"79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3"}
May 25 14:00:41.066379 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:41.066279    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf","Type":"ContainerStarted","Data":"17c28cb8409f2713bcacc80667595728478df0db2d849208f311b9f45bea3d5d"}
May 25 14:00:41.081642 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:41.081591    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5" podStartSLOduration=1.430935691 podStartE2EDuration="4.081576914s" podCreationTimestamp="2026-05-25 14:00:37 +0000 UTC" firstStartedPulling="2026-05-25 14:00:38.276903982 +0000 UTC m=+46.111760054" lastFinishedPulling="2026-05-25 14:00:40.927545204 +0000 UTC m=+48.762401277" observedRunningTime="2026-05-25 14:00:41.080796871 +0000 UTC m=+48.915652964" watchObservedRunningTime="2026-05-25 14:00:41.081576914 +0000 UTC m=+48.916433006"
May 25 14:00:41.875895 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:41.875852    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:41.875895 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:41.875899    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:41.880837 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:41.880802    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:42.070128 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:42.070085    2571 generic.go:358] "Generic (PLEG): container finished" podID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerID="79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3" exitCode=0
May 25 14:00:42.070573 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:42.070178    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf","Type":"ContainerDied","Data":"79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3"}
May 25 14:00:42.074416 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:42.074399    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:00:42.994956 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:42.994908    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jqs89"
May 25 14:00:43.078149 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:43.078013    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e6b44271-ffa5-4b01-b294-9e962e15927c","Type":"ContainerStarted","Data":"584a9d3a18dbb68075e461288f1616e2c1b91ab1140ba565e66344fee8634ade"}
May 25 14:00:43.078149 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:43.078070    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e6b44271-ffa5-4b01-b294-9e962e15927c","Type":"ContainerStarted","Data":"4a2ef1fb4531d22cd23345eeaf1a136583f52f7c19db15e11fda98284245ff7e"}
May 25 14:00:43.078149 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:43.078085    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e6b44271-ffa5-4b01-b294-9e962e15927c","Type":"ContainerStarted","Data":"f90417dd6a6996522fcb4b7f168b377e0065cf6ae1291efa39918f91803601ab"}
May 25 14:00:43.078149 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:43.078098    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e6b44271-ffa5-4b01-b294-9e962e15927c","Type":"ContainerStarted","Data":"1fc0f6299afa1409c21c413ccd61f126104dbc1be4e26cbbd51327b82ab53fa1"}
May 25 14:00:43.078149 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:43.078124    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e6b44271-ffa5-4b01-b294-9e962e15927c","Type":"ContainerStarted","Data":"9c8f30ccb3c4750dc48fed1b7c4678be94bb6cb2fb7a7944c3f9b4e7dc7c65ae"}
May 25 14:00:43.079588 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:43.079560    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-665c9c8584-xlxsr" event={"ID":"d5521ae8-eced-4397-8e19-ae7b75b147e5","Type":"ContainerStarted","Data":"de1b7df9c0d0d988f058befcf12ae1f1b1c370fba79d9d75c5d92a48a2891bae"}
May 25 14:00:43.080063 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:43.080040    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-665c9c8584-xlxsr"
May 25 14:00:43.083610 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:43.083583    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q" event={"ID":"95f37b20-7603-4a8a-9e85-72c406d458fa","Type":"ContainerStarted","Data":"5d8912fcd3e8632bca57c4f9ebc413ab8fa12c470228c9d5360f565aae327d97"}
May 25 14:00:43.083707 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:43.083619    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q" event={"ID":"95f37b20-7603-4a8a-9e85-72c406d458fa","Type":"ContainerStarted","Data":"409d649f2b695a911fd8697306553cfc8d9971d53f685ddb38c8603dc8c76c11"}
May 25 14:00:43.083707 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:43.083634    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q" event={"ID":"95f37b20-7603-4a8a-9e85-72c406d458fa","Type":"ContainerStarted","Data":"9e433c339c2760e39cd695472a906a7eb54f6482e5571339f6ff5408cf2c90b2"}
May 25 14:00:43.083864 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:43.083842    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:43.085475 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:43.085443    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-665c9c8584-xlxsr"
May 25 14:00:43.104742 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:43.104685    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-665c9c8584-xlxsr" podStartSLOduration=2.117320655 podStartE2EDuration="5.104664928s" podCreationTimestamp="2026-05-25 14:00:38 +0000 UTC" firstStartedPulling="2026-05-25 14:00:39.326968177 +0000 UTC m=+47.161824251" lastFinishedPulling="2026-05-25 14:00:42.314312439 +0000 UTC m=+50.149168524" observedRunningTime="2026-05-25 14:00:43.101490626 +0000 UTC m=+50.936346719" watchObservedRunningTime="2026-05-25 14:00:43.104664928 +0000 UTC m=+50.939521022"
May 25 14:00:43.151270 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:43.150420    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q" podStartSLOduration=1.940749599 podStartE2EDuration="7.150399453s" podCreationTimestamp="2026-05-25 14:00:36 +0000 UTC" firstStartedPulling="2026-05-25 14:00:37.106161163 +0000 UTC m=+44.941017248" lastFinishedPulling="2026-05-25 14:00:42.315811028 +0000 UTC m=+50.150667102" observedRunningTime="2026-05-25 14:00:43.149060206 +0000 UTC m=+50.983916300" watchObservedRunningTime="2026-05-25 14:00:43.150399453 +0000 UTC m=+50.985255548"
May 25 14:00:44.091017 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:44.090979    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e6b44271-ffa5-4b01-b294-9e962e15927c","Type":"ContainerStarted","Data":"8d1c2304bcd0d6f3b45b5223aa80a8643991c7081b5d179e08c2128047cb7668"}
May 25 14:00:44.098779 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:44.098750    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5f69c45485-shw4q"
May 25 14:00:44.119276 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:44.119176    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.369277232 podStartE2EDuration="10.119155922s" podCreationTimestamp="2026-05-25 14:00:34 +0000 UTC" firstStartedPulling="2026-05-25 14:00:35.531301022 +0000 UTC m=+43.366157093" lastFinishedPulling="2026-05-25 14:00:42.281179699 +0000 UTC m=+50.116035783" observedRunningTime="2026-05-25 14:00:44.118190346 +0000 UTC m=+51.953046468" watchObservedRunningTime="2026-05-25 14:00:44.119155922 +0000 UTC m=+51.954012015"
May 25 14:00:46.104725 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:46.104688    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf","Type":"ContainerStarted","Data":"b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9"}
May 25 14:00:46.104725 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:46.104732    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf","Type":"ContainerStarted","Data":"83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5"}
May 25 14:00:46.105124 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:46.104745    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf","Type":"ContainerStarted","Data":"90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62"}
May 25 14:00:46.105124 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:46.104756    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf","Type":"ContainerStarted","Data":"90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec"}
May 25 14:00:47.110510 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:47.110476    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf","Type":"ContainerStarted","Data":"726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1"}
May 25 14:00:47.110510 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:47.110511    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf","Type":"ContainerStarted","Data":"a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596"}
May 25 14:00:47.143100 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:47.143012    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.967560433 podStartE2EDuration="8.142996116s" podCreationTimestamp="2026-05-25 14:00:39 +0000 UTC" firstStartedPulling="2026-05-25 14:00:42.281385587 +0000 UTC m=+50.116241661" lastFinishedPulling="2026-05-25 14:00:45.456821255 +0000 UTC m=+53.291677344" observedRunningTime="2026-05-25 14:00:47.141374391 +0000 UTC m=+54.976230484" watchObservedRunningTime="2026-05-25 14:00:47.142996116 +0000 UTC m=+54.977852208"
May 25 14:00:48.177233 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:48.177192    2571 configmap.go:193] Couldn't get configMap openshift-monitoring/prometheus-k8s-rulefiles-0: configmap "prometheus-k8s-rulefiles-0" not found
May 25 14:00:48.177741 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:00:48.177305    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-prometheus-k8s-rulefiles-0 podName:73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf nodeName:}" failed. No retries permitted until 2026-05-25 14:00:48.677277156 +0000 UTC m=+56.512133250 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-k8s-rulefiles-0" (UniqueName: "kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-prometheus-k8s-rulefiles-0") pod "prometheus-k8s-0" (UID: "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf") : configmap "prometheus-k8s-rulefiles-0" not found
May 25 14:00:48.704325 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:48.704281    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:48.704325 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:48.704339    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:48.709092 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:48.709066    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:49.120891 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:49.120801    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:00:49.167714 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:49.167681    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b465c95c9-vrtf8"]
May 25 14:00:49.922079 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:49.922037    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:00:51.955843 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:51.955811    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fstv5"
May 25 14:00:58.112104 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:58.112054    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:00:58.112104 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:58.112111    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:00:58.479022 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:58.478971    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9f60fa8-15a3-402e-842e-f1a52b7a3404-metrics-certs\") pod \"network-metrics-daemon-lc2jb\" (UID: \"f9f60fa8-15a3-402e-842e-f1a52b7a3404\") " pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 14:00:58.479207 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:58.479047    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gs7zp\" (UniqueName: \"kubernetes.io/projected/98f00859-da06-410b-8572-c64348ab32d5-kube-api-access-gs7zp\") pod \"network-check-target-fb2gw\" (UID: \"98f00859-da06-410b-8572-c64348ab32d5\") " pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 14:00:58.481704 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:58.481684    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 25 14:00:58.481891 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:58.481879    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 25 14:00:58.491224 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:58.491195    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 25 14:00:58.491507 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:58.491489    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9f60fa8-15a3-402e-842e-f1a52b7a3404-metrics-certs\") pod \"network-metrics-daemon-lc2jb\" (UID: \"f9f60fa8-15a3-402e-842e-f1a52b7a3404\") " pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 14:00:58.502335 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:58.502309    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs7zp\" (UniqueName: \"kubernetes.io/projected/98f00859-da06-410b-8572-c64348ab32d5-kube-api-access-gs7zp\") pod \"network-check-target-fb2gw\" (UID: \"98f00859-da06-410b-8572-c64348ab32d5\") " pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 14:00:58.535690 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:58.535655    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sdv6d\""
May 25 14:00:58.543976 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:58.543914    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc2jb"
May 25 14:00:58.678510 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:58.678470    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lc2jb"]
May 25 14:00:58.681427 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:00:58.681399    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9f60fa8_15a3_402e_842e_f1a52b7a3404.slice/crio-cd41a81c690fbb9e40db99ade3576e4d4133c3f1c912b70dc652f1ca6e6a4b70 WatchSource:0}: Error finding container cd41a81c690fbb9e40db99ade3576e4d4133c3f1c912b70dc652f1ca6e6a4b70: Status 404 returned error can't find the container with id cd41a81c690fbb9e40db99ade3576e4d4133c3f1c912b70dc652f1ca6e6a4b70
May 25 14:00:58.735130 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:58.735044    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zr45f\""
May 25 14:00:58.743090 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:58.743064    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 14:00:58.872020 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:58.871972    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fb2gw"]
May 25 14:00:58.876255 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:00:58.876225    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98f00859_da06_410b_8572_c64348ab32d5.slice/crio-16b0bfff7f33bc849628c5a952bef9ce9439c90971315c9894c970002d371dba WatchSource:0}: Error finding container 16b0bfff7f33bc849628c5a952bef9ce9439c90971315c9894c970002d371dba: Status 404 returned error can't find the container with id 16b0bfff7f33bc849628c5a952bef9ce9439c90971315c9894c970002d371dba
May 25 14:00:59.146679 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:59.146588    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fb2gw" event={"ID":"98f00859-da06-410b-8572-c64348ab32d5","Type":"ContainerStarted","Data":"16b0bfff7f33bc849628c5a952bef9ce9439c90971315c9894c970002d371dba"}
May 25 14:00:59.147652 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:00:59.147629    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lc2jb" event={"ID":"f9f60fa8-15a3-402e-842e-f1a52b7a3404","Type":"ContainerStarted","Data":"cd41a81c690fbb9e40db99ade3576e4d4133c3f1c912b70dc652f1ca6e6a4b70"}
May 25 14:01:00.152601 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:00.152500    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lc2jb" event={"ID":"f9f60fa8-15a3-402e-842e-f1a52b7a3404","Type":"ContainerStarted","Data":"75fba3b8d7ac6f530be29a33b5adc2c6be2d7276d99c537a08cae89210e7a5ff"}
May 25 14:01:00.152601 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:00.152544    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lc2jb" event={"ID":"f9f60fa8-15a3-402e-842e-f1a52b7a3404","Type":"ContainerStarted","Data":"631d484605aab4c75a94612791e4f689016941f2355ccb4d5bd555c9b2651f7b"}
May 25 14:01:00.170504 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:00.170440    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lc2jb" podStartSLOduration=67.000634886 podStartE2EDuration="1m8.17041715s" podCreationTimestamp="2026-05-25 13:59:52 +0000 UTC" firstStartedPulling="2026-05-25 14:00:58.683501973 +0000 UTC m=+66.518358058" lastFinishedPulling="2026-05-25 14:00:59.853284248 +0000 UTC m=+67.688140322" observedRunningTime="2026-05-25 14:01:00.16846561 +0000 UTC m=+68.003321701" watchObservedRunningTime="2026-05-25 14:01:00.17041715 +0000 UTC m=+68.005273245"
May 25 14:01:02.162355 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:02.162264    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fb2gw" event={"ID":"98f00859-da06-410b-8572-c64348ab32d5","Type":"ContainerStarted","Data":"80113f790f48bb2f2624b88cc4247e3039cf518b22a9fffc3f23731ab9924472"}
May 25 14:01:02.162793 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:02.162381    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 14:01:02.179680 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:02.179614    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fb2gw" podStartSLOduration=67.334532689 podStartE2EDuration="1m10.17959453s" podCreationTimestamp="2026-05-25 13:59:52 +0000 UTC" firstStartedPulling="2026-05-25 14:00:58.878247627 +0000 UTC m=+66.713103697" lastFinishedPulling="2026-05-25 14:01:01.723309458 +0000 UTC m=+69.558165538" observedRunningTime="2026-05-25 14:01:02.177988683 +0000 UTC m=+70.012844776" watchObservedRunningTime="2026-05-25 14:01:02.17959453 +0000 UTC m=+70.014450623"
May 25 14:01:05.082700 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:05.082625    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-84d47c6dcc-6lqfw" podUID="bb3f10fb-2b34-4f98-9f6d-24b94bd20306" containerName="console" containerID="cri-o://8cecd418c55b19964b09ddeafe68ee2b2810670e70095f6f108f08d9f9361f25" gracePeriod=15
May 25 14:01:05.361655 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:05.361631    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-84d47c6dcc-6lqfw_bb3f10fb-2b34-4f98-9f6d-24b94bd20306/console/0.log"
May 25 14:01:05.361795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:05.361709    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84d47c6dcc-6lqfw"
May 25 14:01:05.453367 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:05.453318    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-oauth-serving-cert\") pod \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\" (UID: \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\") "
May 25 14:01:05.453559 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:05.453383    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-console-oauth-config\") pod \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\" (UID: \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\") "
May 25 14:01:05.453559 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:05.453423    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-console-config\") pod \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\" (UID: \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\") "
May 25 14:01:05.453559 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:05.453450    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-console-serving-cert\") pod \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\" (UID: \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\") "
May 25 14:01:05.453559 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:05.453497    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-service-ca\") pod \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\" (UID: \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\") "
May 25 14:01:05.453559 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:05.453520    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8ct5\" (UniqueName: \"kubernetes.io/projected/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-kube-api-access-z8ct5\") pod \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\" (UID: \"bb3f10fb-2b34-4f98-9f6d-24b94bd20306\") "
May 25 14:01:05.453920 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:05.453780    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bb3f10fb-2b34-4f98-9f6d-24b94bd20306" (UID: "bb3f10fb-2b34-4f98-9f6d-24b94bd20306"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:01:05.453920 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:05.453903    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-console-config" (OuterVolumeSpecName: "console-config") pod "bb3f10fb-2b34-4f98-9f6d-24b94bd20306" (UID: "bb3f10fb-2b34-4f98-9f6d-24b94bd20306"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:01:05.454090 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:05.454033    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-service-ca" (OuterVolumeSpecName: "service-ca") pod "bb3f10fb-2b34-4f98-9f6d-24b94bd20306" (UID: "bb3f10fb-2b34-4f98-9f6d-24b94bd20306"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:01:05.455916 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:05.455894    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bb3f10fb-2b34-4f98-9f6d-24b94bd20306" (UID: "bb3f10fb-2b34-4f98-9f6d-24b94bd20306"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:01:05.455916 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:05.455897    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-kube-api-access-z8ct5" (OuterVolumeSpecName: "kube-api-access-z8ct5") pod "bb3f10fb-2b34-4f98-9f6d-24b94bd20306" (UID: "bb3f10fb-2b34-4f98-9f6d-24b94bd20306"). InnerVolumeSpecName "kube-api-access-z8ct5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 14:01:05.456042 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:05.455915    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bb3f10fb-2b34-4f98-9f6d-24b94bd20306" (UID: "bb3f10fb-2b34-4f98-9f6d-24b94bd20306"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:01:05.554492 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:05.554454    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-oauth-serving-cert\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:05.554492 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:05.554484    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-console-oauth-config\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:05.554492 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:05.554494    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-console-config\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:05.554492 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:05.554503    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-console-serving-cert\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:05.554758 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:05.554512    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-service-ca\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:05.554758 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:05.554520    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8ct5\" (UniqueName: \"kubernetes.io/projected/bb3f10fb-2b34-4f98-9f6d-24b94bd20306-kube-api-access-z8ct5\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:06.181729 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:06.181699    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-84d47c6dcc-6lqfw_bb3f10fb-2b34-4f98-9f6d-24b94bd20306/console/0.log"
May 25 14:01:06.182147 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:06.181740    2571 generic.go:358] "Generic (PLEG): container finished" podID="bb3f10fb-2b34-4f98-9f6d-24b94bd20306" containerID="8cecd418c55b19964b09ddeafe68ee2b2810670e70095f6f108f08d9f9361f25" exitCode=2
May 25 14:01:06.182147 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:06.181777    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84d47c6dcc-6lqfw" event={"ID":"bb3f10fb-2b34-4f98-9f6d-24b94bd20306","Type":"ContainerDied","Data":"8cecd418c55b19964b09ddeafe68ee2b2810670e70095f6f108f08d9f9361f25"}
May 25 14:01:06.182147 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:06.181803    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84d47c6dcc-6lqfw"
May 25 14:01:06.182147 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:06.181822    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84d47c6dcc-6lqfw" event={"ID":"bb3f10fb-2b34-4f98-9f6d-24b94bd20306","Type":"ContainerDied","Data":"a5a253001aa3707f3aa501c9a481f026616b307912c63a3d4409af9f1157fc39"}
May 25 14:01:06.182147 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:06.181839    2571 scope.go:117] "RemoveContainer" containerID="8cecd418c55b19964b09ddeafe68ee2b2810670e70095f6f108f08d9f9361f25"
May 25 14:01:06.191124 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:06.191109    2571 scope.go:117] "RemoveContainer" containerID="8cecd418c55b19964b09ddeafe68ee2b2810670e70095f6f108f08d9f9361f25"
May 25 14:01:06.191422 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:01:06.191403    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cecd418c55b19964b09ddeafe68ee2b2810670e70095f6f108f08d9f9361f25\": container with ID starting with 8cecd418c55b19964b09ddeafe68ee2b2810670e70095f6f108f08d9f9361f25 not found: ID does not exist" containerID="8cecd418c55b19964b09ddeafe68ee2b2810670e70095f6f108f08d9f9361f25"
May 25 14:01:06.191468 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:06.191433    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cecd418c55b19964b09ddeafe68ee2b2810670e70095f6f108f08d9f9361f25"} err="failed to get container status \"8cecd418c55b19964b09ddeafe68ee2b2810670e70095f6f108f08d9f9361f25\": rpc error: code = NotFound desc = could not find container \"8cecd418c55b19964b09ddeafe68ee2b2810670e70095f6f108f08d9f9361f25\": container with ID starting with 8cecd418c55b19964b09ddeafe68ee2b2810670e70095f6f108f08d9f9361f25 not found: ID does not exist"
May 25 14:01:06.201769 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:06.201734    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84d47c6dcc-6lqfw"]
May 25 14:01:06.206297 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:06.206275    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84d47c6dcc-6lqfw"]
May 25 14:01:06.826202 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:06.826166    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb3f10fb-2b34-4f98-9f6d-24b94bd20306" path="/var/lib/kubelet/pods/bb3f10fb-2b34-4f98-9f6d-24b94bd20306/volumes"
May 25 14:01:14.191806 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:14.191754    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-7b465c95c9-vrtf8" podUID="68d63673-6d67-4c9c-9454-b2050a39c39d" containerName="console" containerID="cri-o://e902793d72c17769a822de5c457b67d551ff75f4221a7f44fcc1b712bdf7779b" gracePeriod=15
May 25 14:01:14.501031 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:14.501008    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7b465c95c9-vrtf8_68d63673-6d67-4c9c-9454-b2050a39c39d/console/0.log"
May 25 14:01:14.501167 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:14.501081    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:01:14.538512 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:14.538474    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68d63673-6d67-4c9c-9454-b2050a39c39d-service-ca\") pod \"68d63673-6d67-4c9c-9454-b2050a39c39d\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") "
May 25 14:01:14.538682 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:14.538532    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hk8j\" (UniqueName: \"kubernetes.io/projected/68d63673-6d67-4c9c-9454-b2050a39c39d-kube-api-access-8hk8j\") pod \"68d63673-6d67-4c9c-9454-b2050a39c39d\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") "
May 25 14:01:14.538682 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:14.538645    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68d63673-6d67-4c9c-9454-b2050a39c39d-console-serving-cert\") pod \"68d63673-6d67-4c9c-9454-b2050a39c39d\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") "
May 25 14:01:14.538799 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:14.538700    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68d63673-6d67-4c9c-9454-b2050a39c39d-oauth-serving-cert\") pod \"68d63673-6d67-4c9c-9454-b2050a39c39d\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") "
May 25 14:01:14.538799 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:14.538725    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68d63673-6d67-4c9c-9454-b2050a39c39d-console-config\") pod \"68d63673-6d67-4c9c-9454-b2050a39c39d\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") "
May 25 14:01:14.538893 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:14.538802    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68d63673-6d67-4c9c-9454-b2050a39c39d-trusted-ca-bundle\") pod \"68d63673-6d67-4c9c-9454-b2050a39c39d\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") "
May 25 14:01:14.538893 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:14.538877    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68d63673-6d67-4c9c-9454-b2050a39c39d-console-oauth-config\") pod \"68d63673-6d67-4c9c-9454-b2050a39c39d\" (UID: \"68d63673-6d67-4c9c-9454-b2050a39c39d\") "
May 25 14:01:14.539092 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:14.539040    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68d63673-6d67-4c9c-9454-b2050a39c39d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "68d63673-6d67-4c9c-9454-b2050a39c39d" (UID: "68d63673-6d67-4c9c-9454-b2050a39c39d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:01:14.539494 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:14.539150    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68d63673-6d67-4c9c-9454-b2050a39c39d-console-config" (OuterVolumeSpecName: "console-config") pod "68d63673-6d67-4c9c-9454-b2050a39c39d" (UID: "68d63673-6d67-4c9c-9454-b2050a39c39d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:01:14.539494 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:14.539260    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68d63673-6d67-4c9c-9454-b2050a39c39d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "68d63673-6d67-4c9c-9454-b2050a39c39d" (UID: "68d63673-6d67-4c9c-9454-b2050a39c39d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:01:14.539494 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:14.539269    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68d63673-6d67-4c9c-9454-b2050a39c39d-oauth-serving-cert\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:14.539494 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:14.539281    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68d63673-6d67-4c9c-9454-b2050a39c39d-service-ca" (OuterVolumeSpecName: "service-ca") pod "68d63673-6d67-4c9c-9454-b2050a39c39d" (UID: "68d63673-6d67-4c9c-9454-b2050a39c39d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:01:14.539831 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:14.539315    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68d63673-6d67-4c9c-9454-b2050a39c39d-console-config\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:14.541149 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:14.541114    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68d63673-6d67-4c9c-9454-b2050a39c39d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "68d63673-6d67-4c9c-9454-b2050a39c39d" (UID: "68d63673-6d67-4c9c-9454-b2050a39c39d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:01:14.546952 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:14.546887    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68d63673-6d67-4c9c-9454-b2050a39c39d-kube-api-access-8hk8j" (OuterVolumeSpecName: "kube-api-access-8hk8j") pod "68d63673-6d67-4c9c-9454-b2050a39c39d" (UID: "68d63673-6d67-4c9c-9454-b2050a39c39d"). InnerVolumeSpecName "kube-api-access-8hk8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 14:01:14.547669 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:14.547645    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68d63673-6d67-4c9c-9454-b2050a39c39d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "68d63673-6d67-4c9c-9454-b2050a39c39d" (UID: "68d63673-6d67-4c9c-9454-b2050a39c39d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:01:14.640526 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:14.640489    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68d63673-6d67-4c9c-9454-b2050a39c39d-trusted-ca-bundle\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:14.640526 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:14.640525    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68d63673-6d67-4c9c-9454-b2050a39c39d-console-oauth-config\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:14.640526 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:14.640537    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68d63673-6d67-4c9c-9454-b2050a39c39d-service-ca\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:14.640773 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:14.640571    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hk8j\" (UniqueName: \"kubernetes.io/projected/68d63673-6d67-4c9c-9454-b2050a39c39d-kube-api-access-8hk8j\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:14.640773 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:14.640581    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68d63673-6d67-4c9c-9454-b2050a39c39d-console-serving-cert\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:15.211373 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:15.211348    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7b465c95c9-vrtf8_68d63673-6d67-4c9c-9454-b2050a39c39d/console/0.log"
May 25 14:01:15.211850 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:15.211390    2571 generic.go:358] "Generic (PLEG): container finished" podID="68d63673-6d67-4c9c-9454-b2050a39c39d" containerID="e902793d72c17769a822de5c457b67d551ff75f4221a7f44fcc1b712bdf7779b" exitCode=2
May 25 14:01:15.211850 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:15.211428    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b465c95c9-vrtf8" event={"ID":"68d63673-6d67-4c9c-9454-b2050a39c39d","Type":"ContainerDied","Data":"e902793d72c17769a822de5c457b67d551ff75f4221a7f44fcc1b712bdf7779b"}
May 25 14:01:15.211850 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:15.211471    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b465c95c9-vrtf8" event={"ID":"68d63673-6d67-4c9c-9454-b2050a39c39d","Type":"ContainerDied","Data":"4ddac8977ce104a38f2a40c86349cbf76ce1d338ee6a04a0e9eeb5fd6a0e91c8"}
May 25 14:01:15.211850 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:15.211487    2571 scope.go:117] "RemoveContainer" containerID="e902793d72c17769a822de5c457b67d551ff75f4221a7f44fcc1b712bdf7779b"
May 25 14:01:15.211850 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:15.211504    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b465c95c9-vrtf8"
May 25 14:01:15.219945 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:15.219904    2571 scope.go:117] "RemoveContainer" containerID="e902793d72c17769a822de5c457b67d551ff75f4221a7f44fcc1b712bdf7779b"
May 25 14:01:15.220250 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:01:15.220230    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e902793d72c17769a822de5c457b67d551ff75f4221a7f44fcc1b712bdf7779b\": container with ID starting with e902793d72c17769a822de5c457b67d551ff75f4221a7f44fcc1b712bdf7779b not found: ID does not exist" containerID="e902793d72c17769a822de5c457b67d551ff75f4221a7f44fcc1b712bdf7779b"
May 25 14:01:15.220293 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:15.220269    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e902793d72c17769a822de5c457b67d551ff75f4221a7f44fcc1b712bdf7779b"} err="failed to get container status \"e902793d72c17769a822de5c457b67d551ff75f4221a7f44fcc1b712bdf7779b\": rpc error: code = NotFound desc = could not find container \"e902793d72c17769a822de5c457b67d551ff75f4221a7f44fcc1b712bdf7779b\": container with ID starting with e902793d72c17769a822de5c457b67d551ff75f4221a7f44fcc1b712bdf7779b not found: ID does not exist"
May 25 14:01:15.226413 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:15.226384    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b465c95c9-vrtf8"]
May 25 14:01:15.232406 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:15.232381    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b465c95c9-vrtf8"]
May 25 14:01:16.826012 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:16.825980    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68d63673-6d67-4c9c-9454-b2050a39c39d" path="/var/lib/kubelet/pods/68d63673-6d67-4c9c-9454-b2050a39c39d/volumes"
May 25 14:01:18.117121 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:18.117089    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:01:18.121108 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:18.121086    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-65898b7c8b-68bw5"
May 25 14:01:28.677788 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:28.677746    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:28.698287 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:28.698252    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:29.269288 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:29.269261    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:33.170617 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:33.170587    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fb2gw"
May 25 14:01:33.727819 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:33.727777    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 25 14:01:33.728338 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:33.728280    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerName="alertmanager" containerID="cri-o://9c8f30ccb3c4750dc48fed1b7c4678be94bb6cb2fb7a7944c3f9b4e7dc7c65ae" gracePeriod=120
May 25 14:01:33.728433 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:33.728340    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerName="kube-rbac-proxy-metric" containerID="cri-o://584a9d3a18dbb68075e461288f1616e2c1b91ab1140ba565e66344fee8634ade" gracePeriod=120
May 25 14:01:33.728433 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:33.728373    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerName="kube-rbac-proxy-web" containerID="cri-o://f90417dd6a6996522fcb4b7f168b377e0065cf6ae1291efa39918f91803601ab" gracePeriod=120
May 25 14:01:33.728559 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:33.728379    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerName="prom-label-proxy" containerID="cri-o://8d1c2304bcd0d6f3b45b5223aa80a8643991c7081b5d179e08c2128047cb7668" gracePeriod=120
May 25 14:01:33.728559 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:33.728471    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerName="config-reloader" containerID="cri-o://1fc0f6299afa1409c21c413ccd61f126104dbc1be4e26cbbd51327b82ab53fa1" gracePeriod=120
May 25 14:01:33.728559 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:33.728484    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerName="kube-rbac-proxy" containerID="cri-o://4a2ef1fb4531d22cd23345eeaf1a136583f52f7c19db15e11fda98284245ff7e" gracePeriod=120
May 25 14:01:34.271872 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:34.271839    2571 generic.go:358] "Generic (PLEG): container finished" podID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerID="8d1c2304bcd0d6f3b45b5223aa80a8643991c7081b5d179e08c2128047cb7668" exitCode=0
May 25 14:01:34.271872 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:34.271865    2571 generic.go:358] "Generic (PLEG): container finished" podID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerID="4a2ef1fb4531d22cd23345eeaf1a136583f52f7c19db15e11fda98284245ff7e" exitCode=0
May 25 14:01:34.271872 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:34.271875    2571 generic.go:358] "Generic (PLEG): container finished" podID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerID="1fc0f6299afa1409c21c413ccd61f126104dbc1be4e26cbbd51327b82ab53fa1" exitCode=0
May 25 14:01:34.271872 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:34.271882    2571 generic.go:358] "Generic (PLEG): container finished" podID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerID="9c8f30ccb3c4750dc48fed1b7c4678be94bb6cb2fb7a7944c3f9b4e7dc7c65ae" exitCode=0
May 25 14:01:34.272372 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:34.271908    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e6b44271-ffa5-4b01-b294-9e962e15927c","Type":"ContainerDied","Data":"8d1c2304bcd0d6f3b45b5223aa80a8643991c7081b5d179e08c2128047cb7668"}
May 25 14:01:34.272372 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:34.271960    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e6b44271-ffa5-4b01-b294-9e962e15927c","Type":"ContainerDied","Data":"4a2ef1fb4531d22cd23345eeaf1a136583f52f7c19db15e11fda98284245ff7e"}
May 25 14:01:34.272372 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:34.271973    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e6b44271-ffa5-4b01-b294-9e962e15927c","Type":"ContainerDied","Data":"1fc0f6299afa1409c21c413ccd61f126104dbc1be4e26cbbd51327b82ab53fa1"}
May 25 14:01:34.272372 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:34.271981    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e6b44271-ffa5-4b01-b294-9e962e15927c","Type":"ContainerDied","Data":"9c8f30ccb3c4750dc48fed1b7c4678be94bb6cb2fb7a7944c3f9b4e7dc7c65ae"}
May 25 14:01:34.969457 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:34.969431    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.135551 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.135451    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6b44271-ffa5-4b01-b294-9e962e15927c-metrics-client-ca\") pod \"e6b44271-ffa5-4b01-b294-9e962e15927c\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") "
May 25 14:01:35.135551 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.135497    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-kube-rbac-proxy\") pod \"e6b44271-ffa5-4b01-b294-9e962e15927c\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") "
May 25 14:01:35.135551 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.135521    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-config-volume\") pod \"e6b44271-ffa5-4b01-b294-9e962e15927c\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") "
May 25 14:01:35.135551 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.135544    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"e6b44271-ffa5-4b01-b294-9e962e15927c\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") "
May 25 14:01:35.135910 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.135677    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-kube-rbac-proxy-web\") pod \"e6b44271-ffa5-4b01-b294-9e962e15927c\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") "
May 25 14:01:35.135910 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.135726    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pffbv\" (UniqueName: \"kubernetes.io/projected/e6b44271-ffa5-4b01-b294-9e962e15927c-kube-api-access-pffbv\") pod \"e6b44271-ffa5-4b01-b294-9e962e15927c\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") "
May 25 14:01:35.135910 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.135761    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e6b44271-ffa5-4b01-b294-9e962e15927c-config-out\") pod \"e6b44271-ffa5-4b01-b294-9e962e15927c\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") "
May 25 14:01:35.135910 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.135794    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-web-config\") pod \"e6b44271-ffa5-4b01-b294-9e962e15927c\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") "
May 25 14:01:35.135910 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.135845    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-cluster-tls-config\") pod \"e6b44271-ffa5-4b01-b294-9e962e15927c\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") "
May 25 14:01:35.135910 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.135892    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e6b44271-ffa5-4b01-b294-9e962e15927c-tls-assets\") pod \"e6b44271-ffa5-4b01-b294-9e962e15927c\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") "
May 25 14:01:35.136234 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.135963    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e6b44271-ffa5-4b01-b294-9e962e15927c-alertmanager-main-db\") pod \"e6b44271-ffa5-4b01-b294-9e962e15927c\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") "
May 25 14:01:35.136234 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.135999    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-main-tls\") pod \"e6b44271-ffa5-4b01-b294-9e962e15927c\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") "
May 25 14:01:35.136234 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.136054    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6b44271-ffa5-4b01-b294-9e962e15927c-alertmanager-trusted-ca-bundle\") pod \"e6b44271-ffa5-4b01-b294-9e962e15927c\" (UID: \"e6b44271-ffa5-4b01-b294-9e962e15927c\") "
May 25 14:01:35.137150 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.135960    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6b44271-ffa5-4b01-b294-9e962e15927c-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "e6b44271-ffa5-4b01-b294-9e962e15927c" (UID: "e6b44271-ffa5-4b01-b294-9e962e15927c"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:01:35.137150 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.136818    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6b44271-ffa5-4b01-b294-9e962e15927c-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "e6b44271-ffa5-4b01-b294-9e962e15927c" (UID: "e6b44271-ffa5-4b01-b294-9e962e15927c"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:01:35.137150 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.136909    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6b44271-ffa5-4b01-b294-9e962e15927c-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "e6b44271-ffa5-4b01-b294-9e962e15927c" (UID: "e6b44271-ffa5-4b01-b294-9e962e15927c"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 14:01:35.138846 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.138817    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-config-volume" (OuterVolumeSpecName: "config-volume") pod "e6b44271-ffa5-4b01-b294-9e962e15927c" (UID: "e6b44271-ffa5-4b01-b294-9e962e15927c"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:01:35.139195 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.139160    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "e6b44271-ffa5-4b01-b294-9e962e15927c" (UID: "e6b44271-ffa5-4b01-b294-9e962e15927c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:01:35.139195 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.139182    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "e6b44271-ffa5-4b01-b294-9e962e15927c" (UID: "e6b44271-ffa5-4b01-b294-9e962e15927c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:01:35.139345 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.139225    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6b44271-ffa5-4b01-b294-9e962e15927c-kube-api-access-pffbv" (OuterVolumeSpecName: "kube-api-access-pffbv") pod "e6b44271-ffa5-4b01-b294-9e962e15927c" (UID: "e6b44271-ffa5-4b01-b294-9e962e15927c"). InnerVolumeSpecName "kube-api-access-pffbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 14:01:35.139345 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.139250    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "e6b44271-ffa5-4b01-b294-9e962e15927c" (UID: "e6b44271-ffa5-4b01-b294-9e962e15927c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:01:35.140059 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.140033    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6b44271-ffa5-4b01-b294-9e962e15927c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e6b44271-ffa5-4b01-b294-9e962e15927c" (UID: "e6b44271-ffa5-4b01-b294-9e962e15927c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 14:01:35.140142 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.140099    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "e6b44271-ffa5-4b01-b294-9e962e15927c" (UID: "e6b44271-ffa5-4b01-b294-9e962e15927c"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:01:35.140310 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.140282    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6b44271-ffa5-4b01-b294-9e962e15927c-config-out" (OuterVolumeSpecName: "config-out") pod "e6b44271-ffa5-4b01-b294-9e962e15927c" (UID: "e6b44271-ffa5-4b01-b294-9e962e15927c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 14:01:35.144045 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.144020    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "e6b44271-ffa5-4b01-b294-9e962e15927c" (UID: "e6b44271-ffa5-4b01-b294-9e962e15927c"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:01:35.150479 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.150458    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-web-config" (OuterVolumeSpecName: "web-config") pod "e6b44271-ffa5-4b01-b294-9e962e15927c" (UID: "e6b44271-ffa5-4b01-b294-9e962e15927c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:01:35.237356 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.237317    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6b44271-ffa5-4b01-b294-9e962e15927c-metrics-client-ca\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:35.237356 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.237353    2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:35.237356 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.237363    2571 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-config-volume\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:35.237581 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.237373    2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:35.237581 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.237383    2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:35.237581 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.237393    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pffbv\" (UniqueName: \"kubernetes.io/projected/e6b44271-ffa5-4b01-b294-9e962e15927c-kube-api-access-pffbv\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:35.237581 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.237402    2571 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e6b44271-ffa5-4b01-b294-9e962e15927c-config-out\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:35.237581 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.237410    2571 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-web-config\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:35.237581 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.237418    2571 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-cluster-tls-config\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:35.237581 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.237429    2571 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e6b44271-ffa5-4b01-b294-9e962e15927c-tls-assets\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:35.237581 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.237439    2571 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e6b44271-ffa5-4b01-b294-9e962e15927c-alertmanager-main-db\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:35.237581 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.237447    2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e6b44271-ffa5-4b01-b294-9e962e15927c-secret-alertmanager-main-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:35.237581 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.237455    2571 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6b44271-ffa5-4b01-b294-9e962e15927c-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:35.277457 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.277425    2571 generic.go:358] "Generic (PLEG): container finished" podID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerID="584a9d3a18dbb68075e461288f1616e2c1b91ab1140ba565e66344fee8634ade" exitCode=0
May 25 14:01:35.277457 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.277451    2571 generic.go:358] "Generic (PLEG): container finished" podID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerID="f90417dd6a6996522fcb4b7f168b377e0065cf6ae1291efa39918f91803601ab" exitCode=0
May 25 14:01:35.277901 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.277528    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.277901 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.277525    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e6b44271-ffa5-4b01-b294-9e962e15927c","Type":"ContainerDied","Data":"584a9d3a18dbb68075e461288f1616e2c1b91ab1140ba565e66344fee8634ade"}
May 25 14:01:35.277901 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.277567    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e6b44271-ffa5-4b01-b294-9e962e15927c","Type":"ContainerDied","Data":"f90417dd6a6996522fcb4b7f168b377e0065cf6ae1291efa39918f91803601ab"}
May 25 14:01:35.277901 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.277579    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e6b44271-ffa5-4b01-b294-9e962e15927c","Type":"ContainerDied","Data":"ae7c26fa1d4012ce371127553acb294d3d164e8d7f3e2f5ff595f6261e27d699"}
May 25 14:01:35.277901 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.277594    2571 scope.go:117] "RemoveContainer" containerID="8d1c2304bcd0d6f3b45b5223aa80a8643991c7081b5d179e08c2128047cb7668"
May 25 14:01:35.286265 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.286200    2571 scope.go:117] "RemoveContainer" containerID="584a9d3a18dbb68075e461288f1616e2c1b91ab1140ba565e66344fee8634ade"
May 25 14:01:35.293845 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.293825    2571 scope.go:117] "RemoveContainer" containerID="4a2ef1fb4531d22cd23345eeaf1a136583f52f7c19db15e11fda98284245ff7e"
May 25 14:01:35.301593 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.301567    2571 scope.go:117] "RemoveContainer" containerID="f90417dd6a6996522fcb4b7f168b377e0065cf6ae1291efa39918f91803601ab"
May 25 14:01:35.301854 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.301836    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 25 14:01:35.306693 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.306660    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 25 14:01:35.310894 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.310871    2571 scope.go:117] "RemoveContainer" containerID="1fc0f6299afa1409c21c413ccd61f126104dbc1be4e26cbbd51327b82ab53fa1"
May 25 14:01:35.318416 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.318396    2571 scope.go:117] "RemoveContainer" containerID="9c8f30ccb3c4750dc48fed1b7c4678be94bb6cb2fb7a7944c3f9b4e7dc7c65ae"
May 25 14:01:35.326169 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.326139    2571 scope.go:117] "RemoveContainer" containerID="d501924573d849186099b9933c4799ada759a9db9ff5846fecdb8ea3ffed1c5e"
May 25 14:01:35.332487 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.332460    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 25 14:01:35.332799 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.332785    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerName="init-config-reloader"
May 25 14:01:35.332886 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.332805    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerName="init-config-reloader"
May 25 14:01:35.332886 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.332823    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68d63673-6d67-4c9c-9454-b2050a39c39d" containerName="console"
May 25 14:01:35.332886 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.332833    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="68d63673-6d67-4c9c-9454-b2050a39c39d" containerName="console"
May 25 14:01:35.332886 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.332852    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerName="kube-rbac-proxy"
May 25 14:01:35.332886 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.332861    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerName="kube-rbac-proxy"
May 25 14:01:35.332886 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.332870    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerName="prom-label-proxy"
May 25 14:01:35.332886 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.332878    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerName="prom-label-proxy"
May 25 14:01:35.333230 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.332892    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerName="kube-rbac-proxy-metric"
May 25 14:01:35.333230 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.332900    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerName="kube-rbac-proxy-metric"
May 25 14:01:35.333230 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.332909    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb3f10fb-2b34-4f98-9f6d-24b94bd20306" containerName="console"
May 25 14:01:35.333230 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.332917    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb3f10fb-2b34-4f98-9f6d-24b94bd20306" containerName="console"
May 25 14:01:35.333230 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.332946    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerName="alertmanager"
May 25 14:01:35.333230 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.332955    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerName="alertmanager"
May 25 14:01:35.333230 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.332964    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerName="config-reloader"
May 25 14:01:35.333230 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.332972    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerName="config-reloader"
May 25 14:01:35.333230 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.332993    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerName="kube-rbac-proxy-web"
May 25 14:01:35.333230 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.333002    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerName="kube-rbac-proxy-web"
May 25 14:01:35.333230 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.333101    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="bb3f10fb-2b34-4f98-9f6d-24b94bd20306" containerName="console"
May 25 14:01:35.333230 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.333117    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerName="config-reloader"
May 25 14:01:35.333230 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.333130    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerName="kube-rbac-proxy-web"
May 25 14:01:35.333230 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.333141    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerName="kube-rbac-proxy-metric"
May 25 14:01:35.333230 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.333150    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerName="prom-label-proxy"
May 25 14:01:35.333230 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.333159    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerName="alertmanager"
May 25 14:01:35.333230 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.333170    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="68d63673-6d67-4c9c-9454-b2050a39c39d" containerName="console"
May 25 14:01:35.333230 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.333180    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" containerName="kube-rbac-proxy"
May 25 14:01:35.335509 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.335487    2571 scope.go:117] "RemoveContainer" containerID="8d1c2304bcd0d6f3b45b5223aa80a8643991c7081b5d179e08c2128047cb7668"
May 25 14:01:35.336041 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:01:35.335872    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d1c2304bcd0d6f3b45b5223aa80a8643991c7081b5d179e08c2128047cb7668\": container with ID starting with 8d1c2304bcd0d6f3b45b5223aa80a8643991c7081b5d179e08c2128047cb7668 not found: ID does not exist" containerID="8d1c2304bcd0d6f3b45b5223aa80a8643991c7081b5d179e08c2128047cb7668"
May 25 14:01:35.336041 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.335909    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d1c2304bcd0d6f3b45b5223aa80a8643991c7081b5d179e08c2128047cb7668"} err="failed to get container status \"8d1c2304bcd0d6f3b45b5223aa80a8643991c7081b5d179e08c2128047cb7668\": rpc error: code = NotFound desc = could not find container \"8d1c2304bcd0d6f3b45b5223aa80a8643991c7081b5d179e08c2128047cb7668\": container with ID starting with 8d1c2304bcd0d6f3b45b5223aa80a8643991c7081b5d179e08c2128047cb7668 not found: ID does not exist"
May 25 14:01:35.336041 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.335954    2571 scope.go:117] "RemoveContainer" containerID="584a9d3a18dbb68075e461288f1616e2c1b91ab1140ba565e66344fee8634ade"
May 25 14:01:35.336382 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:01:35.336355    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"584a9d3a18dbb68075e461288f1616e2c1b91ab1140ba565e66344fee8634ade\": container with ID starting with 584a9d3a18dbb68075e461288f1616e2c1b91ab1140ba565e66344fee8634ade not found: ID does not exist" containerID="584a9d3a18dbb68075e461288f1616e2c1b91ab1140ba565e66344fee8634ade"
May 25 14:01:35.336450 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.336386    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"584a9d3a18dbb68075e461288f1616e2c1b91ab1140ba565e66344fee8634ade"} err="failed to get container status \"584a9d3a18dbb68075e461288f1616e2c1b91ab1140ba565e66344fee8634ade\": rpc error: code = NotFound desc = could not find container \"584a9d3a18dbb68075e461288f1616e2c1b91ab1140ba565e66344fee8634ade\": container with ID starting with 584a9d3a18dbb68075e461288f1616e2c1b91ab1140ba565e66344fee8634ade not found: ID does not exist"
May 25 14:01:35.336450 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.336410    2571 scope.go:117] "RemoveContainer" containerID="4a2ef1fb4531d22cd23345eeaf1a136583f52f7c19db15e11fda98284245ff7e"
May 25 14:01:35.336713 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:01:35.336690    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a2ef1fb4531d22cd23345eeaf1a136583f52f7c19db15e11fda98284245ff7e\": container with ID starting with 4a2ef1fb4531d22cd23345eeaf1a136583f52f7c19db15e11fda98284245ff7e not found: ID does not exist" containerID="4a2ef1fb4531d22cd23345eeaf1a136583f52f7c19db15e11fda98284245ff7e"
May 25 14:01:35.336770 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.336720    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a2ef1fb4531d22cd23345eeaf1a136583f52f7c19db15e11fda98284245ff7e"} err="failed to get container status \"4a2ef1fb4531d22cd23345eeaf1a136583f52f7c19db15e11fda98284245ff7e\": rpc error: code = NotFound desc = could not find container \"4a2ef1fb4531d22cd23345eeaf1a136583f52f7c19db15e11fda98284245ff7e\": container with ID starting with 4a2ef1fb4531d22cd23345eeaf1a136583f52f7c19db15e11fda98284245ff7e not found: ID does not exist"
May 25 14:01:35.336770 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.336744    2571 scope.go:117] "RemoveContainer" containerID="f90417dd6a6996522fcb4b7f168b377e0065cf6ae1291efa39918f91803601ab"
May 25 14:01:35.337074 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.337056    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.337589 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:01:35.337563    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f90417dd6a6996522fcb4b7f168b377e0065cf6ae1291efa39918f91803601ab\": container with ID starting with f90417dd6a6996522fcb4b7f168b377e0065cf6ae1291efa39918f91803601ab not found: ID does not exist" containerID="f90417dd6a6996522fcb4b7f168b377e0065cf6ae1291efa39918f91803601ab"
May 25 14:01:35.337678 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.337601    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f90417dd6a6996522fcb4b7f168b377e0065cf6ae1291efa39918f91803601ab"} err="failed to get container status \"f90417dd6a6996522fcb4b7f168b377e0065cf6ae1291efa39918f91803601ab\": rpc error: code = NotFound desc = could not find container \"f90417dd6a6996522fcb4b7f168b377e0065cf6ae1291efa39918f91803601ab\": container with ID starting with f90417dd6a6996522fcb4b7f168b377e0065cf6ae1291efa39918f91803601ab not found: ID does not exist"
May 25 14:01:35.337678 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.337625    2571 scope.go:117] "RemoveContainer" containerID="1fc0f6299afa1409c21c413ccd61f126104dbc1be4e26cbbd51327b82ab53fa1"
May 25 14:01:35.337946 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:01:35.337902    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fc0f6299afa1409c21c413ccd61f126104dbc1be4e26cbbd51327b82ab53fa1\": container with ID starting with 1fc0f6299afa1409c21c413ccd61f126104dbc1be4e26cbbd51327b82ab53fa1 not found: ID does not exist" containerID="1fc0f6299afa1409c21c413ccd61f126104dbc1be4e26cbbd51327b82ab53fa1"
May 25 14:01:35.338042 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.337956    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fc0f6299afa1409c21c413ccd61f126104dbc1be4e26cbbd51327b82ab53fa1"} err="failed to get container status \"1fc0f6299afa1409c21c413ccd61f126104dbc1be4e26cbbd51327b82ab53fa1\": rpc error: code = NotFound desc = could not find container \"1fc0f6299afa1409c21c413ccd61f126104dbc1be4e26cbbd51327b82ab53fa1\": container with ID starting with 1fc0f6299afa1409c21c413ccd61f126104dbc1be4e26cbbd51327b82ab53fa1 not found: ID does not exist"
May 25 14:01:35.338042 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.337978    2571 scope.go:117] "RemoveContainer" containerID="9c8f30ccb3c4750dc48fed1b7c4678be94bb6cb2fb7a7944c3f9b4e7dc7c65ae"
May 25 14:01:35.338308 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:01:35.338283    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c8f30ccb3c4750dc48fed1b7c4678be94bb6cb2fb7a7944c3f9b4e7dc7c65ae\": container with ID starting with 9c8f30ccb3c4750dc48fed1b7c4678be94bb6cb2fb7a7944c3f9b4e7dc7c65ae not found: ID does not exist" containerID="9c8f30ccb3c4750dc48fed1b7c4678be94bb6cb2fb7a7944c3f9b4e7dc7c65ae"
May 25 14:01:35.338346 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.338315    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c8f30ccb3c4750dc48fed1b7c4678be94bb6cb2fb7a7944c3f9b4e7dc7c65ae"} err="failed to get container status \"9c8f30ccb3c4750dc48fed1b7c4678be94bb6cb2fb7a7944c3f9b4e7dc7c65ae\": rpc error: code = NotFound desc = could not find container \"9c8f30ccb3c4750dc48fed1b7c4678be94bb6cb2fb7a7944c3f9b4e7dc7c65ae\": container with ID starting with 9c8f30ccb3c4750dc48fed1b7c4678be94bb6cb2fb7a7944c3f9b4e7dc7c65ae not found: ID does not exist"
May 25 14:01:35.338346 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.338340    2571 scope.go:117] "RemoveContainer" containerID="d501924573d849186099b9933c4799ada759a9db9ff5846fecdb8ea3ffed1c5e"
May 25 14:01:35.338579 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:01:35.338558    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d501924573d849186099b9933c4799ada759a9db9ff5846fecdb8ea3ffed1c5e\": container with ID starting with d501924573d849186099b9933c4799ada759a9db9ff5846fecdb8ea3ffed1c5e not found: ID does not exist" containerID="d501924573d849186099b9933c4799ada759a9db9ff5846fecdb8ea3ffed1c5e"
May 25 14:01:35.338668 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.338582    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d501924573d849186099b9933c4799ada759a9db9ff5846fecdb8ea3ffed1c5e"} err="failed to get container status \"d501924573d849186099b9933c4799ada759a9db9ff5846fecdb8ea3ffed1c5e\": rpc error: code = NotFound desc = could not find container \"d501924573d849186099b9933c4799ada759a9db9ff5846fecdb8ea3ffed1c5e\": container with ID starting with d501924573d849186099b9933c4799ada759a9db9ff5846fecdb8ea3ffed1c5e not found: ID does not exist"
May 25 14:01:35.338668 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.338599    2571 scope.go:117] "RemoveContainer" containerID="8d1c2304bcd0d6f3b45b5223aa80a8643991c7081b5d179e08c2128047cb7668"
May 25 14:01:35.338847 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.338830    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d1c2304bcd0d6f3b45b5223aa80a8643991c7081b5d179e08c2128047cb7668"} err="failed to get container status \"8d1c2304bcd0d6f3b45b5223aa80a8643991c7081b5d179e08c2128047cb7668\": rpc error: code = NotFound desc = could not find container \"8d1c2304bcd0d6f3b45b5223aa80a8643991c7081b5d179e08c2128047cb7668\": container with ID starting with 8d1c2304bcd0d6f3b45b5223aa80a8643991c7081b5d179e08c2128047cb7668 not found: ID does not exist"
May 25 14:01:35.338900 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.338849    2571 scope.go:117] "RemoveContainer" containerID="584a9d3a18dbb68075e461288f1616e2c1b91ab1140ba565e66344fee8634ade"
May 25 14:01:35.339135 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.339099    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"584a9d3a18dbb68075e461288f1616e2c1b91ab1140ba565e66344fee8634ade"} err="failed to get container status \"584a9d3a18dbb68075e461288f1616e2c1b91ab1140ba565e66344fee8634ade\": rpc error: code = NotFound desc = could not find container \"584a9d3a18dbb68075e461288f1616e2c1b91ab1140ba565e66344fee8634ade\": container with ID starting with 584a9d3a18dbb68075e461288f1616e2c1b91ab1140ba565e66344fee8634ade not found: ID does not exist"
May 25 14:01:35.339240 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.339135    2571 scope.go:117] "RemoveContainer" containerID="4a2ef1fb4531d22cd23345eeaf1a136583f52f7c19db15e11fda98284245ff7e"
May 25 14:01:35.339399 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.339378    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a2ef1fb4531d22cd23345eeaf1a136583f52f7c19db15e11fda98284245ff7e"} err="failed to get container status \"4a2ef1fb4531d22cd23345eeaf1a136583f52f7c19db15e11fda98284245ff7e\": rpc error: code = NotFound desc = could not find container \"4a2ef1fb4531d22cd23345eeaf1a136583f52f7c19db15e11fda98284245ff7e\": container with ID starting with 4a2ef1fb4531d22cd23345eeaf1a136583f52f7c19db15e11fda98284245ff7e not found: ID does not exist"
May 25 14:01:35.339454 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.339402    2571 scope.go:117] "RemoveContainer" containerID="f90417dd6a6996522fcb4b7f168b377e0065cf6ae1291efa39918f91803601ab"
May 25 14:01:35.339576 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.339561    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
May 25 14:01:35.339642 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.339562    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
May 25 14:01:35.339642 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.339613    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
May 25 14:01:35.339738 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.339668    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-4zn45\""
May 25 14:01:35.339738 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.339562    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
May 25 14:01:35.339738 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.339562    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
May 25 14:01:35.339738 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.339710    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
May 25 14:01:35.339920 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.339615    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
May 25 14:01:35.339920 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.339818    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f90417dd6a6996522fcb4b7f168b377e0065cf6ae1291efa39918f91803601ab"} err="failed to get container status \"f90417dd6a6996522fcb4b7f168b377e0065cf6ae1291efa39918f91803601ab\": rpc error: code = NotFound desc = could not find container \"f90417dd6a6996522fcb4b7f168b377e0065cf6ae1291efa39918f91803601ab\": container with ID starting with f90417dd6a6996522fcb4b7f168b377e0065cf6ae1291efa39918f91803601ab not found: ID does not exist"
May 25 14:01:35.339920 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.339836    2571 scope.go:117] "RemoveContainer" containerID="1fc0f6299afa1409c21c413ccd61f126104dbc1be4e26cbbd51327b82ab53fa1"
May 25 14:01:35.340127 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.340094    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fc0f6299afa1409c21c413ccd61f126104dbc1be4e26cbbd51327b82ab53fa1"} err="failed to get container status \"1fc0f6299afa1409c21c413ccd61f126104dbc1be4e26cbbd51327b82ab53fa1\": rpc error: code = NotFound desc = could not find container \"1fc0f6299afa1409c21c413ccd61f126104dbc1be4e26cbbd51327b82ab53fa1\": container with ID starting with 1fc0f6299afa1409c21c413ccd61f126104dbc1be4e26cbbd51327b82ab53fa1 not found: ID does not exist"
May 25 14:01:35.340127 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.340113    2571 scope.go:117] "RemoveContainer" containerID="9c8f30ccb3c4750dc48fed1b7c4678be94bb6cb2fb7a7944c3f9b4e7dc7c65ae"
May 25 14:01:35.340127 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.340118    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
May 25 14:01:35.340322 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.340296    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c8f30ccb3c4750dc48fed1b7c4678be94bb6cb2fb7a7944c3f9b4e7dc7c65ae"} err="failed to get container status \"9c8f30ccb3c4750dc48fed1b7c4678be94bb6cb2fb7a7944c3f9b4e7dc7c65ae\": rpc error: code = NotFound desc = could not find container \"9c8f30ccb3c4750dc48fed1b7c4678be94bb6cb2fb7a7944c3f9b4e7dc7c65ae\": container with ID starting with 9c8f30ccb3c4750dc48fed1b7c4678be94bb6cb2fb7a7944c3f9b4e7dc7c65ae not found: ID does not exist"
May 25 14:01:35.340425 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.340324    2571 scope.go:117] "RemoveContainer" containerID="d501924573d849186099b9933c4799ada759a9db9ff5846fecdb8ea3ffed1c5e"
May 25 14:01:35.340594 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.340566    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d501924573d849186099b9933c4799ada759a9db9ff5846fecdb8ea3ffed1c5e"} err="failed to get container status \"d501924573d849186099b9933c4799ada759a9db9ff5846fecdb8ea3ffed1c5e\": rpc error: code = NotFound desc = could not find container \"d501924573d849186099b9933c4799ada759a9db9ff5846fecdb8ea3ffed1c5e\": container with ID starting with d501924573d849186099b9933c4799ada759a9db9ff5846fecdb8ea3ffed1c5e not found: ID does not exist"
May 25 14:01:35.344843 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.344824    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
May 25 14:01:35.353912 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.353885    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 25 14:01:35.438636 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.438608    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f28e36a1-7faa-4ece-b092-aebf4ea1f218-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.438794 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.438653    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f28e36a1-7faa-4ece-b092-aebf4ea1f218-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.438794 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.438681    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f28e36a1-7faa-4ece-b092-aebf4ea1f218-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.438794 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.438759    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f28e36a1-7faa-4ece-b092-aebf4ea1f218-config-volume\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.438794 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.438788    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f28e36a1-7faa-4ece-b092-aebf4ea1f218-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.439002 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.438825    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f28e36a1-7faa-4ece-b092-aebf4ea1f218-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.439002 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.438885    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f28e36a1-7faa-4ece-b092-aebf4ea1f218-config-out\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.439002 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.438971    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f28e36a1-7faa-4ece-b092-aebf4ea1f218-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.439098 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.439004    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f28e36a1-7faa-4ece-b092-aebf4ea1f218-web-config\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.439098 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.439023    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f28e36a1-7faa-4ece-b092-aebf4ea1f218-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.439098 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.439042    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc685\" (UniqueName: \"kubernetes.io/projected/f28e36a1-7faa-4ece-b092-aebf4ea1f218-kube-api-access-hc685\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.439098 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.439070    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f28e36a1-7faa-4ece-b092-aebf4ea1f218-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.439098 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.439091    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f28e36a1-7faa-4ece-b092-aebf4ea1f218-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.540018 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.539964    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f28e36a1-7faa-4ece-b092-aebf4ea1f218-web-config\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.540181 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.540024    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f28e36a1-7faa-4ece-b092-aebf4ea1f218-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.540181 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.540054    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hc685\" (UniqueName: \"kubernetes.io/projected/f28e36a1-7faa-4ece-b092-aebf4ea1f218-kube-api-access-hc685\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.540181 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.540083    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f28e36a1-7faa-4ece-b092-aebf4ea1f218-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.540282 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.540214    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f28e36a1-7faa-4ece-b092-aebf4ea1f218-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.540282 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.540259    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f28e36a1-7faa-4ece-b092-aebf4ea1f218-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.540369 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.540299    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f28e36a1-7faa-4ece-b092-aebf4ea1f218-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.540369 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.540330    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f28e36a1-7faa-4ece-b092-aebf4ea1f218-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.540511 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.540383    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f28e36a1-7faa-4ece-b092-aebf4ea1f218-config-volume\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.540511 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.540408    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f28e36a1-7faa-4ece-b092-aebf4ea1f218-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.541114 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.540760    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f28e36a1-7faa-4ece-b092-aebf4ea1f218-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.541236 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.541173    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f28e36a1-7faa-4ece-b092-aebf4ea1f218-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.541236 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.541198    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f28e36a1-7faa-4ece-b092-aebf4ea1f218-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.541236 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.541226    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f28e36a1-7faa-4ece-b092-aebf4ea1f218-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.541381 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.541267    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f28e36a1-7faa-4ece-b092-aebf4ea1f218-config-out\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.541381 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.541314    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f28e36a1-7faa-4ece-b092-aebf4ea1f218-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.543526 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.543451    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f28e36a1-7faa-4ece-b092-aebf4ea1f218-web-config\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.543526 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.543449    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f28e36a1-7faa-4ece-b092-aebf4ea1f218-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.543526 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.543506    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f28e36a1-7faa-4ece-b092-aebf4ea1f218-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.543738 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.543615    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f28e36a1-7faa-4ece-b092-aebf4ea1f218-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.543738 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.543625    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f28e36a1-7faa-4ece-b092-aebf4ea1f218-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.544000 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.543974    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f28e36a1-7faa-4ece-b092-aebf4ea1f218-config-volume\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.544236 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.544218    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f28e36a1-7faa-4ece-b092-aebf4ea1f218-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.544303 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.544263    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f28e36a1-7faa-4ece-b092-aebf4ea1f218-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.545314 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.545295    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f28e36a1-7faa-4ece-b092-aebf4ea1f218-config-out\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.547261 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.547244    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc685\" (UniqueName: \"kubernetes.io/projected/f28e36a1-7faa-4ece-b092-aebf4ea1f218-kube-api-access-hc685\") pod \"alertmanager-main-0\" (UID: \"f28e36a1-7faa-4ece-b092-aebf4ea1f218\") " pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.647097 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.647060    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 25 14:01:35.799916 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:35.799881    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 25 14:01:36.281697 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:36.281659    2571 generic.go:358] "Generic (PLEG): container finished" podID="f28e36a1-7faa-4ece-b092-aebf4ea1f218" containerID="7eafbf27e469efbd5b3bcdd8a4be1531c8f9beb0d9dd43cb5b3a0bb22c50ab55" exitCode=0
May 25 14:01:36.282143 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:36.281751    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f28e36a1-7faa-4ece-b092-aebf4ea1f218","Type":"ContainerDied","Data":"7eafbf27e469efbd5b3bcdd8a4be1531c8f9beb0d9dd43cb5b3a0bb22c50ab55"}
May 25 14:01:36.282143 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:36.281786    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f28e36a1-7faa-4ece-b092-aebf4ea1f218","Type":"ContainerStarted","Data":"aa6ace22fe1dc99f9e74d1b61c1ced67e1d2df0f66551d52c52decea6e5b0522"}
May 25 14:01:36.825774 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:36.825726    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6b44271-ffa5-4b01-b294-9e962e15927c" path="/var/lib/kubelet/pods/e6b44271-ffa5-4b01-b294-9e962e15927c/volumes"
May 25 14:01:37.288741 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.288704    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f28e36a1-7faa-4ece-b092-aebf4ea1f218","Type":"ContainerStarted","Data":"4794a43ee5e5ae3d6b650044ea7a8ef1c273546482887c066c47c945507729d3"}
May 25 14:01:37.288741 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.288743    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f28e36a1-7faa-4ece-b092-aebf4ea1f218","Type":"ContainerStarted","Data":"c327c9d2562598913aa37ae481ac5110ac8ee9212e1f089275ba508ac62fd8f1"}
May 25 14:01:37.289190 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.288753    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f28e36a1-7faa-4ece-b092-aebf4ea1f218","Type":"ContainerStarted","Data":"027e4fac9366a3712c3ada9406ac11bab451fb691a99d3d6a9c64df74074e3c2"}
May 25 14:01:37.289190 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.288761    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f28e36a1-7faa-4ece-b092-aebf4ea1f218","Type":"ContainerStarted","Data":"dfbd9ba8a00f6c83b3b730109d69c4667ad5941ee9e011a5f016a59db8a78d3b"}
May 25 14:01:37.289190 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.288769    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f28e36a1-7faa-4ece-b092-aebf4ea1f218","Type":"ContainerStarted","Data":"a5cf98b10464c37d05ee111c76e24cea49f6a802236bf330cbe5570fb3e5845d"}
May 25 14:01:37.289190 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.288778    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f28e36a1-7faa-4ece-b092-aebf4ea1f218","Type":"ContainerStarted","Data":"65834666ef19a306d3f9682b1c9b1cc18ef3750254a10041d324f0c1a4c9bf6f"}
May 25 14:01:37.317702 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.317653    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.317634045 podStartE2EDuration="2.317634045s" podCreationTimestamp="2026-05-25 14:01:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 14:01:37.315632561 +0000 UTC m=+105.150488653" watchObservedRunningTime="2026-05-25 14:01:37.317634045 +0000 UTC m=+105.152490142"
May 25 14:01:37.736619 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.736581    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5489765fd6-qklxx"]
May 25 14:01:37.740056 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.740037    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx"
May 25 14:01:37.742232 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.742204    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
May 25 14:01:37.742232 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.742216    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-gmh25\""
May 25 14:01:37.742405 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.742217    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
May 25 14:01:37.742405 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.742270    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
May 25 14:01:37.742649 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.742632    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
May 25 14:01:37.742761 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.742748    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
May 25 14:01:37.747533 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.747518    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
May 25 14:01:37.750273 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.750250    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5489765fd6-qklxx"]
May 25 14:01:37.862039 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.861984    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbfe26d4-cd97-40c4-ab84-aeb546c3f07f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5489765fd6-qklxx\" (UID: \"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f\") " pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx"
May 25 14:01:37.862039 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.862038    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lg88\" (UniqueName: \"kubernetes.io/projected/bbfe26d4-cd97-40c4-ab84-aeb546c3f07f-kube-api-access-7lg88\") pod \"telemeter-client-5489765fd6-qklxx\" (UID: \"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f\") " pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx"
May 25 14:01:37.862299 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.862072    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bbfe26d4-cd97-40c4-ab84-aeb546c3f07f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5489765fd6-qklxx\" (UID: \"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f\") " pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx"
May 25 14:01:37.862299 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.862167    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bbfe26d4-cd97-40c4-ab84-aeb546c3f07f-telemeter-client-tls\") pod \"telemeter-client-5489765fd6-qklxx\" (UID: \"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f\") " pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx"
May 25 14:01:37.862299 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.862206    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bbfe26d4-cd97-40c4-ab84-aeb546c3f07f-federate-client-tls\") pod \"telemeter-client-5489765fd6-qklxx\" (UID: \"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f\") " pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx"
May 25 14:01:37.862299 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.862267    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bbfe26d4-cd97-40c4-ab84-aeb546c3f07f-secret-telemeter-client\") pod \"telemeter-client-5489765fd6-qklxx\" (UID: \"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f\") " pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx"
May 25 14:01:37.862479 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.862326    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbfe26d4-cd97-40c4-ab84-aeb546c3f07f-serving-certs-ca-bundle\") pod \"telemeter-client-5489765fd6-qklxx\" (UID: \"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f\") " pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx"
May 25 14:01:37.862479 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.862359    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbfe26d4-cd97-40c4-ab84-aeb546c3f07f-metrics-client-ca\") pod \"telemeter-client-5489765fd6-qklxx\" (UID: \"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f\") " pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx"
May 25 14:01:37.963890 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.963846    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbfe26d4-cd97-40c4-ab84-aeb546c3f07f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5489765fd6-qklxx\" (UID: \"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f\") " pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx"
May 25 14:01:37.964102 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.963945    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7lg88\" (UniqueName: \"kubernetes.io/projected/bbfe26d4-cd97-40c4-ab84-aeb546c3f07f-kube-api-access-7lg88\") pod \"telemeter-client-5489765fd6-qklxx\" (UID: \"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f\") " pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx"
May 25 14:01:37.964102 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.964017    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bbfe26d4-cd97-40c4-ab84-aeb546c3f07f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5489765fd6-qklxx\" (UID: \"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f\") " pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx"
May 25 14:01:37.964102 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.964058    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bbfe26d4-cd97-40c4-ab84-aeb546c3f07f-telemeter-client-tls\") pod \"telemeter-client-5489765fd6-qklxx\" (UID: \"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f\") " pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx"
May 25 14:01:37.964102 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.964088    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bbfe26d4-cd97-40c4-ab84-aeb546c3f07f-federate-client-tls\") pod \"telemeter-client-5489765fd6-qklxx\" (UID: \"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f\") " pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx"
May 25 14:01:37.964305 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.964180    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bbfe26d4-cd97-40c4-ab84-aeb546c3f07f-secret-telemeter-client\") pod \"telemeter-client-5489765fd6-qklxx\" (UID: \"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f\") " pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx"
May 25 14:01:37.964305 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.964245    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbfe26d4-cd97-40c4-ab84-aeb546c3f07f-serving-certs-ca-bundle\") pod \"telemeter-client-5489765fd6-qklxx\" (UID: \"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f\") " pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx"
May 25 14:01:37.964305 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.964275    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbfe26d4-cd97-40c4-ab84-aeb546c3f07f-metrics-client-ca\") pod \"telemeter-client-5489765fd6-qklxx\" (UID: \"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f\") " pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx"
May 25 14:01:37.965043 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.964958    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbfe26d4-cd97-40c4-ab84-aeb546c3f07f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5489765fd6-qklxx\" (UID: \"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f\") " pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx"
May 25 14:01:37.965167 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.965138    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbfe26d4-cd97-40c4-ab84-aeb546c3f07f-metrics-client-ca\") pod \"telemeter-client-5489765fd6-qklxx\" (UID: \"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f\") " pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx"
May 25 14:01:37.965293 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.965173    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbfe26d4-cd97-40c4-ab84-aeb546c3f07f-serving-certs-ca-bundle\") pod \"telemeter-client-5489765fd6-qklxx\" (UID: \"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f\") " pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx"
May 25 14:01:37.966795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.966762    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bbfe26d4-cd97-40c4-ab84-aeb546c3f07f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5489765fd6-qklxx\" (UID: \"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f\") " pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx"
May 25 14:01:37.966891 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.966828    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bbfe26d4-cd97-40c4-ab84-aeb546c3f07f-telemeter-client-tls\") pod \"telemeter-client-5489765fd6-qklxx\" (UID: \"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f\") " pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx"
May 25 14:01:37.966891 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.966842    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bbfe26d4-cd97-40c4-ab84-aeb546c3f07f-secret-telemeter-client\") pod \"telemeter-client-5489765fd6-qklxx\" (UID: \"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f\") " pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx"
May 25 14:01:37.967034 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.966919    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bbfe26d4-cd97-40c4-ab84-aeb546c3f07f-federate-client-tls\") pod \"telemeter-client-5489765fd6-qklxx\" (UID: \"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f\") " pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx"
May 25 14:01:37.971014 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:37.970993    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lg88\" (UniqueName: \"kubernetes.io/projected/bbfe26d4-cd97-40c4-ab84-aeb546c3f07f-kube-api-access-7lg88\") pod \"telemeter-client-5489765fd6-qklxx\" (UID: \"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f\") " pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx"
May 25 14:01:38.006736 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.006637    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 25 14:01:38.007338 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.007158    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerName="prometheus" containerID="cri-o://90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec" gracePeriod=600
May 25 14:01:38.007338 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.007177    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerName="kube-rbac-proxy" containerID="cri-o://726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1" gracePeriod=600
May 25 14:01:38.007338 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.007202    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerName="kube-rbac-proxy-web" containerID="cri-o://b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9" gracePeriod=600
May 25 14:01:38.007338 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.007201    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerName="thanos-sidecar" containerID="cri-o://83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5" gracePeriod=600
May 25 14:01:38.007338 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.007217    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerName="config-reloader" containerID="cri-o://90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62" gracePeriod=600
May 25 14:01:38.007338 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.007209    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerName="kube-rbac-proxy-thanos" containerID="cri-o://a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596" gracePeriod=600
May 25 14:01:38.050535 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.050503    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx"
May 25 14:01:38.192501 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.192461    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5489765fd6-qklxx"]
May 25 14:01:38.195480 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:01:38.195448    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbfe26d4_cd97_40c4_ab84_aeb546c3f07f.slice/crio-523aa60ff238214827e0ba2d92b249882d5461368b0c07204bdae84c6eb2a92e WatchSource:0}: Error finding container 523aa60ff238214827e0ba2d92b249882d5461368b0c07204bdae84c6eb2a92e: Status 404 returned error can't find the container with id 523aa60ff238214827e0ba2d92b249882d5461368b0c07204bdae84c6eb2a92e
May 25 14:01:38.256129 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.256107    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.295465 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.295386    2571 generic.go:358] "Generic (PLEG): container finished" podID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerID="a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596" exitCode=0
May 25 14:01:38.295465 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.295415    2571 generic.go:358] "Generic (PLEG): container finished" podID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerID="726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1" exitCode=0
May 25 14:01:38.295465 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.295422    2571 generic.go:358] "Generic (PLEG): container finished" podID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerID="b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9" exitCode=0
May 25 14:01:38.295465 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.295434    2571 generic.go:358] "Generic (PLEG): container finished" podID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerID="83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5" exitCode=0
May 25 14:01:38.295465 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.295442    2571 generic.go:358] "Generic (PLEG): container finished" podID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerID="90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62" exitCode=0
May 25 14:01:38.295465 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.295449    2571 generic.go:358] "Generic (PLEG): container finished" podID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerID="90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec" exitCode=0
May 25 14:01:38.296174 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.295510    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf","Type":"ContainerDied","Data":"a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596"}
May 25 14:01:38.296174 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.295547    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf","Type":"ContainerDied","Data":"726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1"}
May 25 14:01:38.296174 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.295563    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf","Type":"ContainerDied","Data":"b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9"}
May 25 14:01:38.296174 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.295577    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf","Type":"ContainerDied","Data":"83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5"}
May 25 14:01:38.296174 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.295591    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf","Type":"ContainerDied","Data":"90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62"}
May 25 14:01:38.296174 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.295603    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf","Type":"ContainerDied","Data":"90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec"}
May 25 14:01:38.296174 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.295612    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.296174 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.295616    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf","Type":"ContainerDied","Data":"17c28cb8409f2713bcacc80667595728478df0db2d849208f311b9f45bea3d5d"}
May 25 14:01:38.296174 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.295614    2571 scope.go:117] "RemoveContainer" containerID="a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596"
May 25 14:01:38.297384 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.297337    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx" event={"ID":"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f","Type":"ContainerStarted","Data":"523aa60ff238214827e0ba2d92b249882d5461368b0c07204bdae84c6eb2a92e"}
May 25 14:01:38.303920 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.303894    2571 scope.go:117] "RemoveContainer" containerID="726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1"
May 25 14:01:38.311407 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.311383    2571 scope.go:117] "RemoveContainer" containerID="b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9"
May 25 14:01:38.320725 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.320707    2571 scope.go:117] "RemoveContainer" containerID="83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5"
May 25 14:01:38.327821 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.327789    2571 scope.go:117] "RemoveContainer" containerID="90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62"
May 25 14:01:38.335184 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.335167    2571 scope.go:117] "RemoveContainer" containerID="90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec"
May 25 14:01:38.342531 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.342507    2571 scope.go:117] "RemoveContainer" containerID="79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3"
May 25 14:01:38.349057 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.349036    2571 scope.go:117] "RemoveContainer" containerID="a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596"
May 25 14:01:38.349339 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:01:38.349321    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596\": container with ID starting with a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596 not found: ID does not exist" containerID="a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596"
May 25 14:01:38.349404 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.349348    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596"} err="failed to get container status \"a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596\": rpc error: code = NotFound desc = could not find container \"a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596\": container with ID starting with a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596 not found: ID does not exist"
May 25 14:01:38.349404 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.349367    2571 scope.go:117] "RemoveContainer" containerID="726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1"
May 25 14:01:38.349614 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:01:38.349597    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1\": container with ID starting with 726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1 not found: ID does not exist" containerID="726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1"
May 25 14:01:38.349662 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.349617    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1"} err="failed to get container status \"726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1\": rpc error: code = NotFound desc = could not find container \"726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1\": container with ID starting with 726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1 not found: ID does not exist"
May 25 14:01:38.349662 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.349630    2571 scope.go:117] "RemoveContainer" containerID="b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9"
May 25 14:01:38.349823 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:01:38.349806    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9\": container with ID starting with b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9 not found: ID does not exist" containerID="b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9"
May 25 14:01:38.349866 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.349829    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9"} err="failed to get container status \"b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9\": rpc error: code = NotFound desc = could not find container \"b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9\": container with ID starting with b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9 not found: ID does not exist"
May 25 14:01:38.349866 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.349842    2571 scope.go:117] "RemoveContainer" containerID="83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5"
May 25 14:01:38.350136 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:01:38.350117    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5\": container with ID starting with 83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5 not found: ID does not exist" containerID="83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5"
May 25 14:01:38.350216 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.350137    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5"} err="failed to get container status \"83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5\": rpc error: code = NotFound desc = could not find container \"83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5\": container with ID starting with 83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5 not found: ID does not exist"
May 25 14:01:38.350216 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.350151    2571 scope.go:117] "RemoveContainer" containerID="90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62"
May 25 14:01:38.350361 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:01:38.350348    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62\": container with ID starting with 90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62 not found: ID does not exist" containerID="90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62"
May 25 14:01:38.350393 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.350364    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62"} err="failed to get container status \"90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62\": rpc error: code = NotFound desc = could not find container \"90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62\": container with ID starting with 90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62 not found: ID does not exist"
May 25 14:01:38.350393 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.350375    2571 scope.go:117] "RemoveContainer" containerID="90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec"
May 25 14:01:38.350582 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:01:38.350567    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec\": container with ID starting with 90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec not found: ID does not exist" containerID="90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec"
May 25 14:01:38.350614 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.350585    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec"} err="failed to get container status \"90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec\": rpc error: code = NotFound desc = could not find container \"90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec\": container with ID starting with 90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec not found: ID does not exist"
May 25 14:01:38.350614 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.350598    2571 scope.go:117] "RemoveContainer" containerID="79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3"
May 25 14:01:38.350849 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:01:38.350829    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3\": container with ID starting with 79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3 not found: ID does not exist" containerID="79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3"
May 25 14:01:38.350943 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.350853    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3"} err="failed to get container status \"79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3\": rpc error: code = NotFound desc = could not find container \"79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3\": container with ID starting with 79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3 not found: ID does not exist"
May 25 14:01:38.350943 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.350870    2571 scope.go:117] "RemoveContainer" containerID="a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596"
May 25 14:01:38.351213 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.351190    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596"} err="failed to get container status \"a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596\": rpc error: code = NotFound desc = could not find container \"a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596\": container with ID starting with a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596 not found: ID does not exist"
May 25 14:01:38.351280 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.351213    2571 scope.go:117] "RemoveContainer" containerID="726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1"
May 25 14:01:38.351405 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.351389    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1"} err="failed to get container status \"726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1\": rpc error: code = NotFound desc = could not find container \"726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1\": container with ID starting with 726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1 not found: ID does not exist"
May 25 14:01:38.351443 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.351405    2571 scope.go:117] "RemoveContainer" containerID="b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9"
May 25 14:01:38.351627 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.351607    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9"} err="failed to get container status \"b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9\": rpc error: code = NotFound desc = could not find container \"b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9\": container with ID starting with b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9 not found: ID does not exist"
May 25 14:01:38.351627 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.351626    2571 scope.go:117] "RemoveContainer" containerID="83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5"
May 25 14:01:38.351835 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.351819    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5"} err="failed to get container status \"83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5\": rpc error: code = NotFound desc = could not find container \"83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5\": container with ID starting with 83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5 not found: ID does not exist"
May 25 14:01:38.351881 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.351836    2571 scope.go:117] "RemoveContainer" containerID="90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62"
May 25 14:01:38.352045 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.352029    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62"} err="failed to get container status \"90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62\": rpc error: code = NotFound desc = could not find container \"90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62\": container with ID starting with 90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62 not found: ID does not exist"
May 25 14:01:38.352096 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.352045    2571 scope.go:117] "RemoveContainer" containerID="90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec"
May 25 14:01:38.352234 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.352219    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec"} err="failed to get container status \"90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec\": rpc error: code = NotFound desc = could not find container \"90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec\": container with ID starting with 90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec not found: ID does not exist"
May 25 14:01:38.352268 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.352234    2571 scope.go:117] "RemoveContainer" containerID="79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3"
May 25 14:01:38.352449 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.352430    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3"} err="failed to get container status \"79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3\": rpc error: code = NotFound desc = could not find container \"79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3\": container with ID starting with 79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3 not found: ID does not exist"
May 25 14:01:38.352483 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.352450    2571 scope.go:117] "RemoveContainer" containerID="a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596"
May 25 14:01:38.352662 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.352641    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596"} err="failed to get container status \"a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596\": rpc error: code = NotFound desc = could not find container \"a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596\": container with ID starting with a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596 not found: ID does not exist"
May 25 14:01:38.352701 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.352664    2571 scope.go:117] "RemoveContainer" containerID="726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1"
May 25 14:01:38.352858 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.352841    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1"} err="failed to get container status \"726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1\": rpc error: code = NotFound desc = could not find container \"726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1\": container with ID starting with 726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1 not found: ID does not exist"
May 25 14:01:38.352898 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.352858    2571 scope.go:117] "RemoveContainer" containerID="b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9"
May 25 14:01:38.353076 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.353061    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9"} err="failed to get container status \"b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9\": rpc error: code = NotFound desc = could not find container \"b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9\": container with ID starting with b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9 not found: ID does not exist"
May 25 14:01:38.353134 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.353077    2571 scope.go:117] "RemoveContainer" containerID="83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5"
May 25 14:01:38.353258 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.353243    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5"} err="failed to get container status \"83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5\": rpc error: code = NotFound desc = could not find container \"83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5\": container with ID starting with 83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5 not found: ID does not exist"
May 25 14:01:38.353258 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.353256    2571 scope.go:117] "RemoveContainer" containerID="90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62"
May 25 14:01:38.353420 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.353406    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62"} err="failed to get container status \"90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62\": rpc error: code = NotFound desc = could not find container \"90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62\": container with ID starting with 90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62 not found: ID does not exist"
May 25 14:01:38.353420 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.353419    2571 scope.go:117] "RemoveContainer" containerID="90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec"
May 25 14:01:38.353627 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.353610    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec"} err="failed to get container status \"90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec\": rpc error: code = NotFound desc = could not find container \"90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec\": container with ID starting with 90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec not found: ID does not exist"
May 25 14:01:38.353676 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.353628    2571 scope.go:117] "RemoveContainer" containerID="79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3"
May 25 14:01:38.353834 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.353815    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3"} err="failed to get container status \"79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3\": rpc error: code = NotFound desc = could not find container \"79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3\": container with ID starting with 79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3 not found: ID does not exist"
May 25 14:01:38.353876 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.353834    2571 scope.go:117] "RemoveContainer" containerID="a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596"
May 25 14:01:38.354079 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.354057    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596"} err="failed to get container status \"a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596\": rpc error: code = NotFound desc = could not find container \"a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596\": container with ID starting with a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596 not found: ID does not exist"
May 25 14:01:38.354131 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.354079    2571 scope.go:117] "RemoveContainer" containerID="726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1"
May 25 14:01:38.354286 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.354271    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1"} err="failed to get container status \"726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1\": rpc error: code = NotFound desc = could not find container \"726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1\": container with ID starting with 726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1 not found: ID does not exist"
May 25 14:01:38.354323 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.354287    2571 scope.go:117] "RemoveContainer" containerID="b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9"
May 25 14:01:38.354498 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.354482    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9"} err="failed to get container status \"b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9\": rpc error: code = NotFound desc = could not find container \"b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9\": container with ID starting with b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9 not found: ID does not exist"
May 25 14:01:38.354498 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.354496    2571 scope.go:117] "RemoveContainer" containerID="83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5"
May 25 14:01:38.354669 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.354655    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5"} err="failed to get container status \"83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5\": rpc error: code = NotFound desc = could not find container \"83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5\": container with ID starting with 83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5 not found: ID does not exist"
May 25 14:01:38.354718 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.354669    2571 scope.go:117] "RemoveContainer" containerID="90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62"
May 25 14:01:38.354862 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.354846    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62"} err="failed to get container status \"90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62\": rpc error: code = NotFound desc = could not find container \"90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62\": container with ID starting with 90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62 not found: ID does not exist"
May 25 14:01:38.354901 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.354861    2571 scope.go:117] "RemoveContainer" containerID="90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec"
May 25 14:01:38.355064 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.355048    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec"} err="failed to get container status \"90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec\": rpc error: code = NotFound desc = could not find container \"90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec\": container with ID starting with 90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec not found: ID does not exist"
May 25 14:01:38.355101 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.355065    2571 scope.go:117] "RemoveContainer" containerID="79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3"
May 25 14:01:38.355235 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.355220    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3"} err="failed to get container status \"79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3\": rpc error: code = NotFound desc = could not find container \"79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3\": container with ID starting with 79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3 not found: ID does not exist"
May 25 14:01:38.355282 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.355235    2571 scope.go:117] "RemoveContainer" containerID="a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596"
May 25 14:01:38.355421 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.355407    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596"} err="failed to get container status \"a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596\": rpc error: code = NotFound desc = could not find container \"a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596\": container with ID starting with a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596 not found: ID does not exist"
May 25 14:01:38.355461 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.355422    2571 scope.go:117] "RemoveContainer" containerID="726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1"
May 25 14:01:38.355620 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.355597    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1"} err="failed to get container status \"726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1\": rpc error: code = NotFound desc = could not find container \"726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1\": container with ID starting with 726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1 not found: ID does not exist"
May 25 14:01:38.355695 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.355621    2571 scope.go:117] "RemoveContainer" containerID="b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9"
May 25 14:01:38.355815 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.355799    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9"} err="failed to get container status \"b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9\": rpc error: code = NotFound desc = could not find container \"b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9\": container with ID starting with b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9 not found: ID does not exist"
May 25 14:01:38.355856 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.355816    2571 scope.go:117] "RemoveContainer" containerID="83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5"
May 25 14:01:38.356047 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.356027    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5"} err="failed to get container status \"83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5\": rpc error: code = NotFound desc = could not find container \"83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5\": container with ID starting with 83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5 not found: ID does not exist"
May 25 14:01:38.356092 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.356048    2571 scope.go:117] "RemoveContainer" containerID="90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62"
May 25 14:01:38.356261 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.356245    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62"} err="failed to get container status \"90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62\": rpc error: code = NotFound desc = could not find container \"90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62\": container with ID starting with 90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62 not found: ID does not exist"
May 25 14:01:38.356316 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.356262    2571 scope.go:117] "RemoveContainer" containerID="90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec"
May 25 14:01:38.356491 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.356469    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec"} err="failed to get container status \"90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec\": rpc error: code = NotFound desc = could not find container \"90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec\": container with ID starting with 90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec not found: ID does not exist"
May 25 14:01:38.356535 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.356491    2571 scope.go:117] "RemoveContainer" containerID="79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3"
May 25 14:01:38.356717 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.356695    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3"} err="failed to get container status \"79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3\": rpc error: code = NotFound desc = could not find container \"79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3\": container with ID starting with 79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3 not found: ID does not exist"
May 25 14:01:38.356776 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.356719    2571 scope.go:117] "RemoveContainer" containerID="a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596"
May 25 14:01:38.356912 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.356893    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596"} err="failed to get container status \"a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596\": rpc error: code = NotFound desc = could not find container \"a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596\": container with ID starting with a0bb3e5535c69f58edf3fdaa174fa0af3c15d1ff27c5cde0bc5a1d90cd9ad596 not found: ID does not exist"
May 25 14:01:38.356980 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.356914    2571 scope.go:117] "RemoveContainer" containerID="726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1"
May 25 14:01:38.357147 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.357128    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1"} err="failed to get container status \"726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1\": rpc error: code = NotFound desc = could not find container \"726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1\": container with ID starting with 726dc90df617476289879c97a8e702f586c7a5c0fd0b10828c07b310052bf3a1 not found: ID does not exist"
May 25 14:01:38.357188 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.357147    2571 scope.go:117] "RemoveContainer" containerID="b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9"
May 25 14:01:38.357366 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.357349    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9"} err="failed to get container status \"b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9\": rpc error: code = NotFound desc = could not find container \"b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9\": container with ID starting with b4db30f02f3650f2eaa2f98a4aafa4aaa65b65bcd9887be67534a59bbad0dbb9 not found: ID does not exist"
May 25 14:01:38.357418 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.357367    2571 scope.go:117] "RemoveContainer" containerID="83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5"
May 25 14:01:38.357591 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.357573    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5"} err="failed to get container status \"83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5\": rpc error: code = NotFound desc = could not find container \"83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5\": container with ID starting with 83c7b63872fa9f734eddfe88b68e6c46d8adfd246d5764a72013c9bd63ad3cf5 not found: ID does not exist"
May 25 14:01:38.357636 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.357593    2571 scope.go:117] "RemoveContainer" containerID="90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62"
May 25 14:01:38.357786 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.357772    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62"} err="failed to get container status \"90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62\": rpc error: code = NotFound desc = could not find container \"90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62\": container with ID starting with 90eb23c873508506a0f01c01758f70074302e5dd4573c961363d7cb1b8345e62 not found: ID does not exist"
May 25 14:01:38.357823 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.357786    2571 scope.go:117] "RemoveContainer" containerID="90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec"
May 25 14:01:38.358026 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.358005    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec"} err="failed to get container status \"90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec\": rpc error: code = NotFound desc = could not find container \"90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec\": container with ID starting with 90887afc69bdf5a221977c355a4c2304b347bdb801b7a5662c22bb9d692e57ec not found: ID does not exist"
May 25 14:01:38.358078 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.358026    2571 scope.go:117] "RemoveContainer" containerID="79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3"
May 25 14:01:38.358243 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.358224    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3"} err="failed to get container status \"79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3\": rpc error: code = NotFound desc = could not find container \"79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3\": container with ID starting with 79b34b3fc389e1e6f693206c2981e6c6a61a7cc8c578c4475996f2d46cd5fdc3 not found: ID does not exist"
May 25 14:01:38.369575 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.369544    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-configmap-metrics-client-ca\") pod \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") "
May 25 14:01:38.369678 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.369587    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdxvd\" (UniqueName: \"kubernetes.io/projected/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-kube-api-access-fdxvd\") pod \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") "
May 25 14:01:38.369678 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.369619    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-configmap-serving-certs-ca-bundle\") pod \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") "
May 25 14:01:38.369786 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.369732    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-metrics-client-certs\") pod \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") "
May 25 14:01:38.369839 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.369789    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-kube-rbac-proxy\") pod \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") "
May 25 14:01:38.369839 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.369835    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-prometheus-k8s-tls\") pod \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") "
May 25 14:01:38.369949 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.369865    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-prometheus-k8s-db\") pod \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") "
May 25 14:01:38.369949 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.369909    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-prometheus-k8s-rulefiles-0\") pod \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") "
May 25 14:01:38.370047 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.369976    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-prometheus-trusted-ca-bundle\") pod \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") "
May 25 14:01:38.370047 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.370015    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-web-config\") pod \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") "
May 25 14:01:38.370138 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.370056    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-tls-assets\") pod \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") "
May 25 14:01:38.370138 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.370054    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" (UID: "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:01:38.370138 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.370060    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" (UID: "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:01:38.370138 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.370080    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") "
May 25 14:01:38.370138 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.370109    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-config\") pod \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") "
May 25 14:01:38.370138 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.370132    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-grpc-tls\") pod \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") "
May 25 14:01:38.370413 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.370166    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") "
May 25 14:01:38.370413 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.370198    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-config-out\") pod \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") "
May 25 14:01:38.370413 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.370224    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-configmap-kubelet-serving-ca-bundle\") pod \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") "
May 25 14:01:38.370413 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.370250    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-thanos-prometheus-http-client-file\") pod \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\" (UID: \"73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf\") "
May 25 14:01:38.370608 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.370445    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" (UID: "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:01:38.370608 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.370535    2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:38.370608 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.370554    2571 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-configmap-metrics-client-ca\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:38.370608 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.370570    2571 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:38.371709 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.371663    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" (UID: "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 14:01:38.372320 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.372290    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" (UID: "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:01:38.372417 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.372317    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" (UID: "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:01:38.372519 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.372479    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" (UID: "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:01:38.372925 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.372901    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-kube-api-access-fdxvd" (OuterVolumeSpecName: "kube-api-access-fdxvd") pod "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" (UID: "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf"). InnerVolumeSpecName "kube-api-access-fdxvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 14:01:38.373015 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.372899    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" (UID: "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:01:38.373656 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.373626    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" (UID: "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:01:38.374879 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.374841    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" (UID: "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:01:38.375018 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.374990    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-config" (OuterVolumeSpecName: "config") pod "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" (UID: "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:01:38.375123 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.375045    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" (UID: "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:01:38.375123 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.375070    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" (UID: "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:01:38.375228 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.375127    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-config-out" (OuterVolumeSpecName: "config-out") pod "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" (UID: "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 14:01:38.375228 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.375213    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" (UID: "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 14:01:38.375677 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.375649    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" (UID: "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:01:38.385950 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.385900    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-web-config" (OuterVolumeSpecName: "web-config") pod "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" (UID: "73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:01:38.471337 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.471123    2571 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-web-config\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:38.471337 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.471156    2571 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-tls-assets\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:38.471337 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.471172    2571 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:38.471337 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.471187    2571 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-config\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:38.471337 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.471201    2571 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-grpc-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:38.471337 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.471215    2571 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:38.471337 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.471228    2571 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-config-out\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:38.471337 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.471242    2571 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:38.471337 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.471257    2571 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:38.471337 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.471288    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdxvd\" (UniqueName: \"kubernetes.io/projected/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-kube-api-access-fdxvd\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:38.471337 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.471318    2571 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-metrics-client-certs\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:38.471337 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.471337    2571 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-kube-rbac-proxy\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:38.471337 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.471353    2571 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:38.471337 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.471367    2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-prometheus-k8s-db\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:38.471914 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.471380    2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:01:38.618740 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.618701    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 25 14:01:38.622826 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.622797    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 25 14:01:38.651395 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.651350    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 25 14:01:38.651784 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.651765    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerName="init-config-reloader"
May 25 14:01:38.651887 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.651788    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerName="init-config-reloader"
May 25 14:01:38.651887 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.651800    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerName="thanos-sidecar"
May 25 14:01:38.651887 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.651807    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerName="thanos-sidecar"
May 25 14:01:38.651887 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.651816    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerName="kube-rbac-proxy-web"
May 25 14:01:38.651887 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.651825    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerName="kube-rbac-proxy-web"
May 25 14:01:38.651887 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.651841    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerName="prometheus"
May 25 14:01:38.651887 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.651848    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerName="prometheus"
May 25 14:01:38.651887 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.651861    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerName="kube-rbac-proxy-thanos"
May 25 14:01:38.651887 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.651868    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerName="kube-rbac-proxy-thanos"
May 25 14:01:38.651887 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.651879    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerName="kube-rbac-proxy"
May 25 14:01:38.651887 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.651889    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerName="kube-rbac-proxy"
May 25 14:01:38.652393 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.651910    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerName="config-reloader"
May 25 14:01:38.652393 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.651919    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerName="config-reloader"
May 25 14:01:38.652393 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.652024    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerName="config-reloader"
May 25 14:01:38.652393 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.652037    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerName="kube-rbac-proxy"
May 25 14:01:38.652393 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.652053    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerName="thanos-sidecar"
May 25 14:01:38.652393 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.652063    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerName="kube-rbac-proxy-web"
May 25 14:01:38.652393 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.652074    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerName="kube-rbac-proxy-thanos"
May 25 14:01:38.652393 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.652084    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" containerName="prometheus"
May 25 14:01:38.656259 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.656236    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.658495 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.658471    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
May 25 14:01:38.658602 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.658472    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
May 25 14:01:38.658755 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.658735    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
May 25 14:01:38.658755 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.658750    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
May 25 14:01:38.658872 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.658816    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
May 25 14:01:38.658872 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.658841    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
May 25 14:01:38.659057 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.659042    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
May 25 14:01:38.659289 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.659272    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-lp57l\""
May 25 14:01:38.659349 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.659306    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
May 25 14:01:38.659418 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.659393    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
May 25 14:01:38.659670 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.659651    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
May 25 14:01:38.659731 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.659706    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-br2gdh0mi2rfg\""
May 25 14:01:38.662920 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.662895    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
May 25 14:01:38.668040 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.667970    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 25 14:01:38.668514 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.668491    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
May 25 14:01:38.773466 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.773428    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.773466 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.773479    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.773720 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.773552    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee10d117-ba7d-43c6-9669-79951557238f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.773720 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.773588    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.773720 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.773609    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.773720 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.773624    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee10d117-ba7d-43c6-9669-79951557238f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.773720 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.773651    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee10d117-ba7d-43c6-9669-79951557238f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.773923 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.773740    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee10d117-ba7d-43c6-9669-79951557238f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.773923 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.773769    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ee10d117-ba7d-43c6-9669-79951557238f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.773923 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.773788    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-config\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.773923 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.773808    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.773923 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.773876    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.773923 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.773923    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee10d117-ba7d-43c6-9669-79951557238f-config-out\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.774196 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.773976    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ee10d117-ba7d-43c6-9669-79951557238f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.774196 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.773999    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-web-config\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.774196 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.774024    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76cgq\" (UniqueName: \"kubernetes.io/projected/ee10d117-ba7d-43c6-9669-79951557238f-kube-api-access-76cgq\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.774196 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.774064    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.774196 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.774097    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee10d117-ba7d-43c6-9669-79951557238f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.827331 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.827296    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf" path="/var/lib/kubelet/pods/73a1b7d9-68a3-47a6-a3b2-1f3afb3bebaf/volumes"
May 25 14:01:38.875667 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.875583    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee10d117-ba7d-43c6-9669-79951557238f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.875832 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.875636    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee10d117-ba7d-43c6-9669-79951557238f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.875832 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.875744    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ee10d117-ba7d-43c6-9669-79951557238f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.875832 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.875776    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-config\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.875832 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.875802    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.876065 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.875849    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.876065 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.875889    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee10d117-ba7d-43c6-9669-79951557238f-config-out\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.876065 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.875916    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ee10d117-ba7d-43c6-9669-79951557238f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.876065 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.875966    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-web-config\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.876065 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.876009    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-76cgq\" (UniqueName: \"kubernetes.io/projected/ee10d117-ba7d-43c6-9669-79951557238f-kube-api-access-76cgq\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.877223 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.876392    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee10d117-ba7d-43c6-9669-79951557238f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.877356 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.877244    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ee10d117-ba7d-43c6-9669-79951557238f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.879369 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.879339    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee10d117-ba7d-43c6-9669-79951557238f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.880059 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.880038    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.880239 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.880171    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.880239 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.880177    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-web-config\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.880239 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.880218    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee10d117-ba7d-43c6-9669-79951557238f-config-out\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.880484 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.876040    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.880484 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.880336    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee10d117-ba7d-43c6-9669-79951557238f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.880484 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.880346    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ee10d117-ba7d-43c6-9669-79951557238f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.880484 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.880380    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.880484 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.880406    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.880484 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.880439    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee10d117-ba7d-43c6-9669-79951557238f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.880484 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.880473    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.880866 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.880492    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.880866 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.880509    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee10d117-ba7d-43c6-9669-79951557238f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.881036 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.881009    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee10d117-ba7d-43c6-9669-79951557238f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.881626 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.881583    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee10d117-ba7d-43c6-9669-79951557238f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.881877 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.881852    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-config\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.881967 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.881879    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee10d117-ba7d-43c6-9669-79951557238f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.883315 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.883274    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.883906 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.883805    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.883906 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.883866    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.884129 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.884105    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.884644 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.884623    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ee10d117-ba7d-43c6-9669-79951557238f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.885362 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.885345    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76cgq\" (UniqueName: \"kubernetes.io/projected/ee10d117-ba7d-43c6-9669-79951557238f-kube-api-access-76cgq\") pod \"prometheus-k8s-0\" (UID: \"ee10d117-ba7d-43c6-9669-79951557238f\") " pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:38.966577 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:38.966541    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:39.139969 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:39.139862    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 25 14:01:39.142439 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:01:39.142403    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee10d117_ba7d_43c6_9669_79951557238f.slice/crio-bbc79c5d2b0f45a7dd9caa1ca64ce82915e8cc3561e282ac1d83f7e0ea2386e1 WatchSource:0}: Error finding container bbc79c5d2b0f45a7dd9caa1ca64ce82915e8cc3561e282ac1d83f7e0ea2386e1: Status 404 returned error can't find the container with id bbc79c5d2b0f45a7dd9caa1ca64ce82915e8cc3561e282ac1d83f7e0ea2386e1
May 25 14:01:39.302679 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:39.302643    2571 generic.go:358] "Generic (PLEG): container finished" podID="ee10d117-ba7d-43c6-9669-79951557238f" containerID="aaaa3b35ea9c003edb2e14892cb6dab87caabfe256eb22269d94d0bb100029ca" exitCode=0
May 25 14:01:39.303097 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:39.302709    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee10d117-ba7d-43c6-9669-79951557238f","Type":"ContainerDied","Data":"aaaa3b35ea9c003edb2e14892cb6dab87caabfe256eb22269d94d0bb100029ca"}
May 25 14:01:39.303097 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:39.302745    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee10d117-ba7d-43c6-9669-79951557238f","Type":"ContainerStarted","Data":"bbc79c5d2b0f45a7dd9caa1ca64ce82915e8cc3561e282ac1d83f7e0ea2386e1"}
May 25 14:01:40.309571 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:40.309534    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx" event={"ID":"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f","Type":"ContainerStarted","Data":"f255b53af98ef24b65f7fd3dc3a0be4a70478b3ca2a0b2df71c9e1eb76fd3457"}
May 25 14:01:40.310012 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:40.309577    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx" event={"ID":"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f","Type":"ContainerStarted","Data":"92b96a1d7e9800792f2f514556613e15a82e618e3319677b7a0f070d7e399a84"}
May 25 14:01:40.312260 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:40.312220    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee10d117-ba7d-43c6-9669-79951557238f","Type":"ContainerStarted","Data":"9405d19f3494de94a41c3c779d473dff471744ec5b5a19e93f720c96e8149e73"}
May 25 14:01:40.312260 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:40.312261    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee10d117-ba7d-43c6-9669-79951557238f","Type":"ContainerStarted","Data":"602730365b7b967a5829a47476a5070a589a7b511ace06f3d322a74315209f0d"}
May 25 14:01:40.312414 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:40.312275    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee10d117-ba7d-43c6-9669-79951557238f","Type":"ContainerStarted","Data":"11c18e3aa8c4e271916f883187f90c6cf1608c0ece43fd5d336c76822e19d57a"}
May 25 14:01:40.312414 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:40.312288    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee10d117-ba7d-43c6-9669-79951557238f","Type":"ContainerStarted","Data":"75b1e57727c2d0cdc3bcfea6e70c2956da351d615b8712ecaac794c1dac24043"}
May 25 14:01:40.312414 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:40.312299    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee10d117-ba7d-43c6-9669-79951557238f","Type":"ContainerStarted","Data":"94a6e13c632cad4a10f6264deee001ff55210f5a4ab7c8f7f4b6ede22ee85791"}
May 25 14:01:40.312414 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:40.312312    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee10d117-ba7d-43c6-9669-79951557238f","Type":"ContainerStarted","Data":"e810faa7a49f10dd0055be0e54f820e14266c577553847a824e70c9d4765d03a"}
May 25 14:01:40.352686 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:40.352559    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.352543725 podStartE2EDuration="2.352543725s" podCreationTimestamp="2026-05-25 14:01:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 14:01:40.351611286 +0000 UTC m=+108.186467384" watchObservedRunningTime="2026-05-25 14:01:40.352543725 +0000 UTC m=+108.187399818"
May 25 14:01:41.318076 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:41.318032    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx" event={"ID":"bbfe26d4-cd97-40c4-ab84-aeb546c3f07f","Type":"ContainerStarted","Data":"9e60f50be23c8cd8bc41c4932a3dfb6492632eb9dd25a024e2862ecb9fa000ac"}
May 25 14:01:41.339708 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:41.339655    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5489765fd6-qklxx" podStartSLOduration=2.404033631 podStartE2EDuration="4.339636332s" podCreationTimestamp="2026-05-25 14:01:37 +0000 UTC" firstStartedPulling="2026-05-25 14:01:38.197510359 +0000 UTC m=+106.032366432" lastFinishedPulling="2026-05-25 14:01:40.133113062 +0000 UTC m=+107.967969133" observedRunningTime="2026-05-25 14:01:41.338585213 +0000 UTC m=+109.173441306" watchObservedRunningTime="2026-05-25 14:01:41.339636332 +0000 UTC m=+109.174492426"
May 25 14:01:41.968035 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:41.967999    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-556c75cf48-6hnkd"]
May 25 14:01:41.970743 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:41.970723    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:41.982330 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:41.982293    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-556c75cf48-6hnkd"]
May 25 14:01:42.111279 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:42.111233    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c59cfbc4-b708-45cb-a92c-82c477eee56e-console-serving-cert\") pod \"console-556c75cf48-6hnkd\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") " pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:42.111486 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:42.111299    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c59cfbc4-b708-45cb-a92c-82c477eee56e-trusted-ca-bundle\") pod \"console-556c75cf48-6hnkd\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") " pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:42.111486 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:42.111331    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5bw5\" (UniqueName: \"kubernetes.io/projected/c59cfbc4-b708-45cb-a92c-82c477eee56e-kube-api-access-d5bw5\") pod \"console-556c75cf48-6hnkd\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") " pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:42.111486 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:42.111359    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c59cfbc4-b708-45cb-a92c-82c477eee56e-oauth-serving-cert\") pod \"console-556c75cf48-6hnkd\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") " pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:42.111486 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:42.111382    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c59cfbc4-b708-45cb-a92c-82c477eee56e-console-oauth-config\") pod \"console-556c75cf48-6hnkd\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") " pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:42.111486 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:42.111466    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c59cfbc4-b708-45cb-a92c-82c477eee56e-console-config\") pod \"console-556c75cf48-6hnkd\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") " pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:42.111662 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:42.111520    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c59cfbc4-b708-45cb-a92c-82c477eee56e-service-ca\") pod \"console-556c75cf48-6hnkd\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") " pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:42.212291 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:42.212253    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c59cfbc4-b708-45cb-a92c-82c477eee56e-console-serving-cert\") pod \"console-556c75cf48-6hnkd\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") " pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:42.212470 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:42.212314    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c59cfbc4-b708-45cb-a92c-82c477eee56e-trusted-ca-bundle\") pod \"console-556c75cf48-6hnkd\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") " pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:42.212470 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:42.212342    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d5bw5\" (UniqueName: \"kubernetes.io/projected/c59cfbc4-b708-45cb-a92c-82c477eee56e-kube-api-access-d5bw5\") pod \"console-556c75cf48-6hnkd\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") " pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:42.212470 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:42.212369    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c59cfbc4-b708-45cb-a92c-82c477eee56e-oauth-serving-cert\") pod \"console-556c75cf48-6hnkd\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") " pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:42.212470 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:42.212398    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c59cfbc4-b708-45cb-a92c-82c477eee56e-console-oauth-config\") pod \"console-556c75cf48-6hnkd\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") " pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:42.212648 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:42.212536    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c59cfbc4-b708-45cb-a92c-82c477eee56e-console-config\") pod \"console-556c75cf48-6hnkd\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") " pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:42.212778 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:42.212755    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c59cfbc4-b708-45cb-a92c-82c477eee56e-service-ca\") pod \"console-556c75cf48-6hnkd\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") " pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:42.213186 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:42.213159    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c59cfbc4-b708-45cb-a92c-82c477eee56e-oauth-serving-cert\") pod \"console-556c75cf48-6hnkd\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") " pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:42.213463 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:42.213277    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c59cfbc4-b708-45cb-a92c-82c477eee56e-trusted-ca-bundle\") pod \"console-556c75cf48-6hnkd\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") " pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:42.213463 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:42.213328    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c59cfbc4-b708-45cb-a92c-82c477eee56e-service-ca\") pod \"console-556c75cf48-6hnkd\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") " pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:42.213573 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:42.213463    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c59cfbc4-b708-45cb-a92c-82c477eee56e-console-config\") pod \"console-556c75cf48-6hnkd\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") " pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:42.215043 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:42.215011    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c59cfbc4-b708-45cb-a92c-82c477eee56e-console-oauth-config\") pod \"console-556c75cf48-6hnkd\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") " pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:42.215196 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:42.215176    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c59cfbc4-b708-45cb-a92c-82c477eee56e-console-serving-cert\") pod \"console-556c75cf48-6hnkd\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") " pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:42.222589 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:42.222528    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5bw5\" (UniqueName: \"kubernetes.io/projected/c59cfbc4-b708-45cb-a92c-82c477eee56e-kube-api-access-d5bw5\") pod \"console-556c75cf48-6hnkd\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") " pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:42.280487 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:42.280440    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:42.414409 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:42.414300    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-556c75cf48-6hnkd"]
May 25 14:01:42.417108 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:01:42.417078    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc59cfbc4_b708_45cb_a92c_82c477eee56e.slice/crio-a89cbfb776069bfeb93a1278ea4d57ab354b8460ee36ec64ced54b9384896f3b WatchSource:0}: Error finding container a89cbfb776069bfeb93a1278ea4d57ab354b8460ee36ec64ced54b9384896f3b: Status 404 returned error can't find the container with id a89cbfb776069bfeb93a1278ea4d57ab354b8460ee36ec64ced54b9384896f3b
May 25 14:01:43.326891 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:43.326855    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-556c75cf48-6hnkd" event={"ID":"c59cfbc4-b708-45cb-a92c-82c477eee56e","Type":"ContainerStarted","Data":"48fb52c815d1c366c676bbb563957983cea3b3bad85d8325418ff7383647add7"}
May 25 14:01:43.326891 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:43.326894    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-556c75cf48-6hnkd" event={"ID":"c59cfbc4-b708-45cb-a92c-82c477eee56e","Type":"ContainerStarted","Data":"a89cbfb776069bfeb93a1278ea4d57ab354b8460ee36ec64ced54b9384896f3b"}
May 25 14:01:43.344320 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:43.344271    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-556c75cf48-6hnkd" podStartSLOduration=2.344254643 podStartE2EDuration="2.344254643s" podCreationTimestamp="2026-05-25 14:01:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 14:01:43.34337259 +0000 UTC m=+111.178228697" watchObservedRunningTime="2026-05-25 14:01:43.344254643 +0000 UTC m=+111.179110736"
May 25 14:01:43.967513 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:43.967473    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:01:52.280996 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:52.280953    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:52.280996 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:52.281004    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:52.285828 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:52.285799    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:52.356877 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:52.356849    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:01:52.412944 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:01:52.412899    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-df8bdcb5b-2gg87"]
May 25 14:02:17.432973 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:17.432891    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-df8bdcb5b-2gg87" podUID="334a1409-7274-4632-9443-6e81f9e2b6e2" containerName="console" containerID="cri-o://207bc0eb010d4280947e91231bd3141e92f4a2314e5b0bbd155e086f721e202e" gracePeriod=15
May 25 14:02:17.680568 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:17.680534    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-df8bdcb5b-2gg87_334a1409-7274-4632-9443-6e81f9e2b6e2/console/0.log"
May 25 14:02:17.680743 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:17.680614    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:02:17.843966 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:17.843903    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/334a1409-7274-4632-9443-6e81f9e2b6e2-service-ca\") pod \"334a1409-7274-4632-9443-6e81f9e2b6e2\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") "
May 25 14:02:17.844186 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:17.844007    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/334a1409-7274-4632-9443-6e81f9e2b6e2-console-config\") pod \"334a1409-7274-4632-9443-6e81f9e2b6e2\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") "
May 25 14:02:17.844186 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:17.844032    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/334a1409-7274-4632-9443-6e81f9e2b6e2-console-oauth-config\") pod \"334a1409-7274-4632-9443-6e81f9e2b6e2\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") "
May 25 14:02:17.844186 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:17.844066    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/334a1409-7274-4632-9443-6e81f9e2b6e2-trusted-ca-bundle\") pod \"334a1409-7274-4632-9443-6e81f9e2b6e2\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") "
May 25 14:02:17.844186 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:17.844099    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/334a1409-7274-4632-9443-6e81f9e2b6e2-console-serving-cert\") pod \"334a1409-7274-4632-9443-6e81f9e2b6e2\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") "
May 25 14:02:17.844186 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:17.844115    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/334a1409-7274-4632-9443-6e81f9e2b6e2-oauth-serving-cert\") pod \"334a1409-7274-4632-9443-6e81f9e2b6e2\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") "
May 25 14:02:17.844186 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:17.844150    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrtx5\" (UniqueName: \"kubernetes.io/projected/334a1409-7274-4632-9443-6e81f9e2b6e2-kube-api-access-wrtx5\") pod \"334a1409-7274-4632-9443-6e81f9e2b6e2\" (UID: \"334a1409-7274-4632-9443-6e81f9e2b6e2\") "
May 25 14:02:17.844494 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:17.844454    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/334a1409-7274-4632-9443-6e81f9e2b6e2-service-ca" (OuterVolumeSpecName: "service-ca") pod "334a1409-7274-4632-9443-6e81f9e2b6e2" (UID: "334a1409-7274-4632-9443-6e81f9e2b6e2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:02:17.844600 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:17.844573    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/334a1409-7274-4632-9443-6e81f9e2b6e2-console-config" (OuterVolumeSpecName: "console-config") pod "334a1409-7274-4632-9443-6e81f9e2b6e2" (UID: "334a1409-7274-4632-9443-6e81f9e2b6e2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:02:17.844654 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:17.844587    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/334a1409-7274-4632-9443-6e81f9e2b6e2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "334a1409-7274-4632-9443-6e81f9e2b6e2" (UID: "334a1409-7274-4632-9443-6e81f9e2b6e2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:02:17.844654 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:17.844598    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/334a1409-7274-4632-9443-6e81f9e2b6e2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "334a1409-7274-4632-9443-6e81f9e2b6e2" (UID: "334a1409-7274-4632-9443-6e81f9e2b6e2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:02:17.846465 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:17.846438    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/334a1409-7274-4632-9443-6e81f9e2b6e2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "334a1409-7274-4632-9443-6e81f9e2b6e2" (UID: "334a1409-7274-4632-9443-6e81f9e2b6e2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:02:17.846465 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:17.846457    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/334a1409-7274-4632-9443-6e81f9e2b6e2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "334a1409-7274-4632-9443-6e81f9e2b6e2" (UID: "334a1409-7274-4632-9443-6e81f9e2b6e2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:02:17.846603 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:17.846508    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/334a1409-7274-4632-9443-6e81f9e2b6e2-kube-api-access-wrtx5" (OuterVolumeSpecName: "kube-api-access-wrtx5") pod "334a1409-7274-4632-9443-6e81f9e2b6e2" (UID: "334a1409-7274-4632-9443-6e81f9e2b6e2"). InnerVolumeSpecName "kube-api-access-wrtx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 14:02:17.945187 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:17.945139    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/334a1409-7274-4632-9443-6e81f9e2b6e2-console-config\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:02:17.945187 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:17.945181    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/334a1409-7274-4632-9443-6e81f9e2b6e2-console-oauth-config\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:02:17.945187 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:17.945194    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/334a1409-7274-4632-9443-6e81f9e2b6e2-trusted-ca-bundle\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:02:17.945187 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:17.945204    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/334a1409-7274-4632-9443-6e81f9e2b6e2-console-serving-cert\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:02:17.945466 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:17.945213    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/334a1409-7274-4632-9443-6e81f9e2b6e2-oauth-serving-cert\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:02:17.945466 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:17.945222    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrtx5\" (UniqueName: \"kubernetes.io/projected/334a1409-7274-4632-9443-6e81f9e2b6e2-kube-api-access-wrtx5\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:02:17.945466 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:17.945230    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/334a1409-7274-4632-9443-6e81f9e2b6e2-service-ca\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:02:18.430911 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:18.430880    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-df8bdcb5b-2gg87_334a1409-7274-4632-9443-6e81f9e2b6e2/console/0.log"
May 25 14:02:18.431137 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:18.430926    2571 generic.go:358] "Generic (PLEG): container finished" podID="334a1409-7274-4632-9443-6e81f9e2b6e2" containerID="207bc0eb010d4280947e91231bd3141e92f4a2314e5b0bbd155e086f721e202e" exitCode=2
May 25 14:02:18.431137 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:18.431020    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-df8bdcb5b-2gg87"
May 25 14:02:18.431137 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:18.431032    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-df8bdcb5b-2gg87" event={"ID":"334a1409-7274-4632-9443-6e81f9e2b6e2","Type":"ContainerDied","Data":"207bc0eb010d4280947e91231bd3141e92f4a2314e5b0bbd155e086f721e202e"}
May 25 14:02:18.431137 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:18.431069    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-df8bdcb5b-2gg87" event={"ID":"334a1409-7274-4632-9443-6e81f9e2b6e2","Type":"ContainerDied","Data":"eca646e4ec1438940affbb767d5742c7274419f904606421275d7a91bc586304"}
May 25 14:02:18.431137 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:18.431089    2571 scope.go:117] "RemoveContainer" containerID="207bc0eb010d4280947e91231bd3141e92f4a2314e5b0bbd155e086f721e202e"
May 25 14:02:18.439677 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:18.439598    2571 scope.go:117] "RemoveContainer" containerID="207bc0eb010d4280947e91231bd3141e92f4a2314e5b0bbd155e086f721e202e"
May 25 14:02:18.439926 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:02:18.439907    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"207bc0eb010d4280947e91231bd3141e92f4a2314e5b0bbd155e086f721e202e\": container with ID starting with 207bc0eb010d4280947e91231bd3141e92f4a2314e5b0bbd155e086f721e202e not found: ID does not exist" containerID="207bc0eb010d4280947e91231bd3141e92f4a2314e5b0bbd155e086f721e202e"
May 25 14:02:18.440070 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:18.440048    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"207bc0eb010d4280947e91231bd3141e92f4a2314e5b0bbd155e086f721e202e"} err="failed to get container status \"207bc0eb010d4280947e91231bd3141e92f4a2314e5b0bbd155e086f721e202e\": rpc error: code = NotFound desc = could not find container \"207bc0eb010d4280947e91231bd3141e92f4a2314e5b0bbd155e086f721e202e\": container with ID starting with 207bc0eb010d4280947e91231bd3141e92f4a2314e5b0bbd155e086f721e202e not found: ID does not exist"
May 25 14:02:18.451418 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:18.451388    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-df8bdcb5b-2gg87"]
May 25 14:02:18.455129 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:18.455100    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-df8bdcb5b-2gg87"]
May 25 14:02:18.826315 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:18.826268    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="334a1409-7274-4632-9443-6e81f9e2b6e2" path="/var/lib/kubelet/pods/334a1409-7274-4632-9443-6e81f9e2b6e2/volumes"
May 25 14:02:38.967023 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:38.966986    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:02:38.983450 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:38.983422    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:02:39.510170 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:39.510139    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
May 25 14:02:52.546899 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.546851    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-57dcd5df8-lv4w2"]
May 25 14:02:52.547309 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.547223    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="334a1409-7274-4632-9443-6e81f9e2b6e2" containerName="console"
May 25 14:02:52.547309 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.547239    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="334a1409-7274-4632-9443-6e81f9e2b6e2" containerName="console"
May 25 14:02:52.547309 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.547301    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="334a1409-7274-4632-9443-6e81f9e2b6e2" containerName="console"
May 25 14:02:52.550129 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.550107    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:02:52.560210 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.560182    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57dcd5df8-lv4w2"]
May 25 14:02:52.654733 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.654683    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-console-serving-cert\") pod \"console-57dcd5df8-lv4w2\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") " pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:02:52.654733 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.654736    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-service-ca\") pod \"console-57dcd5df8-lv4w2\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") " pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:02:52.654965 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.654834    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-console-config\") pod \"console-57dcd5df8-lv4w2\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") " pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:02:52.654965 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.654863    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-trusted-ca-bundle\") pod \"console-57dcd5df8-lv4w2\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") " pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:02:52.654965 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.654920    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-oauth-serving-cert\") pod \"console-57dcd5df8-lv4w2\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") " pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:02:52.655062 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.655007    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2x6t\" (UniqueName: \"kubernetes.io/projected/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-kube-api-access-d2x6t\") pod \"console-57dcd5df8-lv4w2\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") " pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:02:52.655097 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.655060    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-console-oauth-config\") pod \"console-57dcd5df8-lv4w2\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") " pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:02:52.756485 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.756430    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-console-config\") pod \"console-57dcd5df8-lv4w2\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") " pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:02:52.756485 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.756490    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-trusted-ca-bundle\") pod \"console-57dcd5df8-lv4w2\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") " pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:02:52.756732 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.756527    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-oauth-serving-cert\") pod \"console-57dcd5df8-lv4w2\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") " pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:02:52.756732 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.756561    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d2x6t\" (UniqueName: \"kubernetes.io/projected/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-kube-api-access-d2x6t\") pod \"console-57dcd5df8-lv4w2\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") " pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:02:52.756732 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.756582    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-console-oauth-config\") pod \"console-57dcd5df8-lv4w2\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") " pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:02:52.756732 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.756615    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-console-serving-cert\") pod \"console-57dcd5df8-lv4w2\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") " pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:02:52.756732 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.756636    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-service-ca\") pod \"console-57dcd5df8-lv4w2\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") " pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:02:52.757290 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.757265    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-console-config\") pod \"console-57dcd5df8-lv4w2\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") " pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:02:52.757400 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.757377    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-oauth-serving-cert\") pod \"console-57dcd5df8-lv4w2\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") " pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:02:52.757511 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.757466    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-service-ca\") pod \"console-57dcd5df8-lv4w2\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") " pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:02:52.757591 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.757574    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-trusted-ca-bundle\") pod \"console-57dcd5df8-lv4w2\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") " pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:02:52.759193 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.759163    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-console-serving-cert\") pod \"console-57dcd5df8-lv4w2\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") " pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:02:52.759280 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.759171    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-console-oauth-config\") pod \"console-57dcd5df8-lv4w2\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") " pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:02:52.764361 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.764337    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2x6t\" (UniqueName: \"kubernetes.io/projected/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-kube-api-access-d2x6t\") pod \"console-57dcd5df8-lv4w2\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") " pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:02:52.860745 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:52.860629    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:02:53.014527 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:53.014495    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57dcd5df8-lv4w2"]
May 25 14:02:53.017136 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:02:53.017101    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1707709d_e6b6_40f0_8167_6bbcd4bbb4ea.slice/crio-936671e6a6f2ee51c6ca04330d2324043a5701e8197637a716fead5a09a6e92c WatchSource:0}: Error finding container 936671e6a6f2ee51c6ca04330d2324043a5701e8197637a716fead5a09a6e92c: Status 404 returned error can't find the container with id 936671e6a6f2ee51c6ca04330d2324043a5701e8197637a716fead5a09a6e92c
May 25 14:02:53.536674 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:53.536636    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57dcd5df8-lv4w2" event={"ID":"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea","Type":"ContainerStarted","Data":"6254e7acd60dbb585032bb60dd594fd4003032168d8c6b5105702041e4fdef2a"}
May 25 14:02:53.536674 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:53.536675    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57dcd5df8-lv4w2" event={"ID":"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea","Type":"ContainerStarted","Data":"936671e6a6f2ee51c6ca04330d2324043a5701e8197637a716fead5a09a6e92c"}
May 25 14:02:53.557224 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:02:53.557149    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-57dcd5df8-lv4w2" podStartSLOduration=1.557131507 podStartE2EDuration="1.557131507s" podCreationTimestamp="2026-05-25 14:02:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 14:02:53.555257068 +0000 UTC m=+181.390113161" watchObservedRunningTime="2026-05-25 14:02:53.557131507 +0000 UTC m=+181.391987600"
May 25 14:03:02.861511 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:02.861447    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:03:02.861511 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:02.861509    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:03:02.866370 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:02.866341    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:03:03.572423 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:03.572391    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:03:03.618763 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:03.618723    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-556c75cf48-6hnkd"]
May 25 14:03:28.642286 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:28.642228    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-556c75cf48-6hnkd" podUID="c59cfbc4-b708-45cb-a92c-82c477eee56e" containerName="console" containerID="cri-o://48fb52c815d1c366c676bbb563957983cea3b3bad85d8325418ff7383647add7" gracePeriod=15
May 25 14:03:28.883418 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:28.883388    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-556c75cf48-6hnkd_c59cfbc4-b708-45cb-a92c-82c477eee56e/console/0.log"
May 25 14:03:28.883549 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:28.883452    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:03:28.997136 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:28.997100    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c59cfbc4-b708-45cb-a92c-82c477eee56e-service-ca\") pod \"c59cfbc4-b708-45cb-a92c-82c477eee56e\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") "
May 25 14:03:28.997335 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:28.997150    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c59cfbc4-b708-45cb-a92c-82c477eee56e-oauth-serving-cert\") pod \"c59cfbc4-b708-45cb-a92c-82c477eee56e\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") "
May 25 14:03:28.997335 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:28.997176    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c59cfbc4-b708-45cb-a92c-82c477eee56e-trusted-ca-bundle\") pod \"c59cfbc4-b708-45cb-a92c-82c477eee56e\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") "
May 25 14:03:28.997335 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:28.997204    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5bw5\" (UniqueName: \"kubernetes.io/projected/c59cfbc4-b708-45cb-a92c-82c477eee56e-kube-api-access-d5bw5\") pod \"c59cfbc4-b708-45cb-a92c-82c477eee56e\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") "
May 25 14:03:28.997872 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:28.997320    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c59cfbc4-b708-45cb-a92c-82c477eee56e-console-config\") pod \"c59cfbc4-b708-45cb-a92c-82c477eee56e\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") "
May 25 14:03:28.997872 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:28.997771    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c59cfbc4-b708-45cb-a92c-82c477eee56e-console-serving-cert\") pod \"c59cfbc4-b708-45cb-a92c-82c477eee56e\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") "
May 25 14:03:28.997872 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:28.997854    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c59cfbc4-b708-45cb-a92c-82c477eee56e-console-oauth-config\") pod \"c59cfbc4-b708-45cb-a92c-82c477eee56e\" (UID: \"c59cfbc4-b708-45cb-a92c-82c477eee56e\") "
May 25 14:03:28.997872 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:28.997850    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c59cfbc4-b708-45cb-a92c-82c477eee56e-service-ca" (OuterVolumeSpecName: "service-ca") pod "c59cfbc4-b708-45cb-a92c-82c477eee56e" (UID: "c59cfbc4-b708-45cb-a92c-82c477eee56e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:03:28.998143 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:28.997866    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c59cfbc4-b708-45cb-a92c-82c477eee56e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c59cfbc4-b708-45cb-a92c-82c477eee56e" (UID: "c59cfbc4-b708-45cb-a92c-82c477eee56e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:03:28.998143 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:28.997893    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c59cfbc4-b708-45cb-a92c-82c477eee56e-console-config" (OuterVolumeSpecName: "console-config") pod "c59cfbc4-b708-45cb-a92c-82c477eee56e" (UID: "c59cfbc4-b708-45cb-a92c-82c477eee56e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:03:28.998143 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:28.997991    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c59cfbc4-b708-45cb-a92c-82c477eee56e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c59cfbc4-b708-45cb-a92c-82c477eee56e" (UID: "c59cfbc4-b708-45cb-a92c-82c477eee56e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:03:29.000475 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:29.000437    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c59cfbc4-b708-45cb-a92c-82c477eee56e-kube-api-access-d5bw5" (OuterVolumeSpecName: "kube-api-access-d5bw5") pod "c59cfbc4-b708-45cb-a92c-82c477eee56e" (UID: "c59cfbc4-b708-45cb-a92c-82c477eee56e"). InnerVolumeSpecName "kube-api-access-d5bw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 14:03:29.001990 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:29.000578    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c59cfbc4-b708-45cb-a92c-82c477eee56e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c59cfbc4-b708-45cb-a92c-82c477eee56e" (UID: "c59cfbc4-b708-45cb-a92c-82c477eee56e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:03:29.001990 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:29.000753    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c59cfbc4-b708-45cb-a92c-82c477eee56e-oauth-serving-cert\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:03:29.001990 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:29.000775    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c59cfbc4-b708-45cb-a92c-82c477eee56e-trusted-ca-bundle\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:03:29.001990 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:29.000790    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5bw5\" (UniqueName: \"kubernetes.io/projected/c59cfbc4-b708-45cb-a92c-82c477eee56e-kube-api-access-d5bw5\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:03:29.001990 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:29.000803    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c59cfbc4-b708-45cb-a92c-82c477eee56e-console-config\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:03:29.001990 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:29.000824    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c59cfbc4-b708-45cb-a92c-82c477eee56e-console-oauth-config\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:03:29.001990 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:29.000838    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c59cfbc4-b708-45cb-a92c-82c477eee56e-service-ca\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:03:29.005613 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:29.005545    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c59cfbc4-b708-45cb-a92c-82c477eee56e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c59cfbc4-b708-45cb-a92c-82c477eee56e" (UID: "c59cfbc4-b708-45cb-a92c-82c477eee56e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:03:29.101778 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:29.101741    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c59cfbc4-b708-45cb-a92c-82c477eee56e-console-serving-cert\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:03:29.644135 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:29.644110    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-556c75cf48-6hnkd_c59cfbc4-b708-45cb-a92c-82c477eee56e/console/0.log"
May 25 14:03:29.644619 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:29.644150    2571 generic.go:358] "Generic (PLEG): container finished" podID="c59cfbc4-b708-45cb-a92c-82c477eee56e" containerID="48fb52c815d1c366c676bbb563957983cea3b3bad85d8325418ff7383647add7" exitCode=2
May 25 14:03:29.644619 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:29.644191    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-556c75cf48-6hnkd" event={"ID":"c59cfbc4-b708-45cb-a92c-82c477eee56e","Type":"ContainerDied","Data":"48fb52c815d1c366c676bbb563957983cea3b3bad85d8325418ff7383647add7"}
May 25 14:03:29.644619 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:29.644234    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-556c75cf48-6hnkd" event={"ID":"c59cfbc4-b708-45cb-a92c-82c477eee56e","Type":"ContainerDied","Data":"a89cbfb776069bfeb93a1278ea4d57ab354b8460ee36ec64ced54b9384896f3b"}
May 25 14:03:29.644619 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:29.644252    2571 scope.go:117] "RemoveContainer" containerID="48fb52c815d1c366c676bbb563957983cea3b3bad85d8325418ff7383647add7"
May 25 14:03:29.644619 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:29.644257    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-556c75cf48-6hnkd"
May 25 14:03:29.652826 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:29.652808    2571 scope.go:117] "RemoveContainer" containerID="48fb52c815d1c366c676bbb563957983cea3b3bad85d8325418ff7383647add7"
May 25 14:03:29.653155 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:03:29.653133    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48fb52c815d1c366c676bbb563957983cea3b3bad85d8325418ff7383647add7\": container with ID starting with 48fb52c815d1c366c676bbb563957983cea3b3bad85d8325418ff7383647add7 not found: ID does not exist" containerID="48fb52c815d1c366c676bbb563957983cea3b3bad85d8325418ff7383647add7"
May 25 14:03:29.653209 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:29.653165    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48fb52c815d1c366c676bbb563957983cea3b3bad85d8325418ff7383647add7"} err="failed to get container status \"48fb52c815d1c366c676bbb563957983cea3b3bad85d8325418ff7383647add7\": rpc error: code = NotFound desc = could not find container \"48fb52c815d1c366c676bbb563957983cea3b3bad85d8325418ff7383647add7\": container with ID starting with 48fb52c815d1c366c676bbb563957983cea3b3bad85d8325418ff7383647add7 not found: ID does not exist"
May 25 14:03:29.664820 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:29.664791    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-556c75cf48-6hnkd"]
May 25 14:03:29.668439 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:29.668410    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-556c75cf48-6hnkd"]
May 25 14:03:30.826446 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:30.826409    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c59cfbc4-b708-45cb-a92c-82c477eee56e" path="/var/lib/kubelet/pods/c59cfbc4-b708-45cb-a92c-82c477eee56e/volumes"
May 25 14:03:31.479252 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:31.479219    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-6w2rv"]
May 25 14:03:31.479600 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:31.479587    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c59cfbc4-b708-45cb-a92c-82c477eee56e" containerName="console"
May 25 14:03:31.479649 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:31.479602    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c59cfbc4-b708-45cb-a92c-82c477eee56e" containerName="console"
May 25 14:03:31.479686 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:31.479677    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="c59cfbc4-b708-45cb-a92c-82c477eee56e" containerName="console"
May 25 14:03:31.481641 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:31.481626    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w2rv"
May 25 14:03:31.483680 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:31.483660    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 25 14:03:31.491555 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:31.491529    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6w2rv"]
May 25 14:03:31.625790 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:31.625754    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f731cec-0664-4e49-891b-53bd4603dbba-original-pull-secret\") pod \"global-pull-secret-syncer-6w2rv\" (UID: \"0f731cec-0664-4e49-891b-53bd4603dbba\") " pod="kube-system/global-pull-secret-syncer-6w2rv"
May 25 14:03:31.625790 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:31.625794    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0f731cec-0664-4e49-891b-53bd4603dbba-kubelet-config\") pod \"global-pull-secret-syncer-6w2rv\" (UID: \"0f731cec-0664-4e49-891b-53bd4603dbba\") " pod="kube-system/global-pull-secret-syncer-6w2rv"
May 25 14:03:31.626094 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:31.625907    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0f731cec-0664-4e49-891b-53bd4603dbba-dbus\") pod \"global-pull-secret-syncer-6w2rv\" (UID: \"0f731cec-0664-4e49-891b-53bd4603dbba\") " pod="kube-system/global-pull-secret-syncer-6w2rv"
May 25 14:03:31.735015 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:31.727438    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0f731cec-0664-4e49-891b-53bd4603dbba-dbus\") pod \"global-pull-secret-syncer-6w2rv\" (UID: \"0f731cec-0664-4e49-891b-53bd4603dbba\") " pod="kube-system/global-pull-secret-syncer-6w2rv"
May 25 14:03:31.735015 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:31.727553    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f731cec-0664-4e49-891b-53bd4603dbba-original-pull-secret\") pod \"global-pull-secret-syncer-6w2rv\" (UID: \"0f731cec-0664-4e49-891b-53bd4603dbba\") " pod="kube-system/global-pull-secret-syncer-6w2rv"
May 25 14:03:31.735015 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:31.727578    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0f731cec-0664-4e49-891b-53bd4603dbba-kubelet-config\") pod \"global-pull-secret-syncer-6w2rv\" (UID: \"0f731cec-0664-4e49-891b-53bd4603dbba\") " pod="kube-system/global-pull-secret-syncer-6w2rv"
May 25 14:03:31.735015 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:31.727687    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0f731cec-0664-4e49-891b-53bd4603dbba-kubelet-config\") pod \"global-pull-secret-syncer-6w2rv\" (UID: \"0f731cec-0664-4e49-891b-53bd4603dbba\") " pod="kube-system/global-pull-secret-syncer-6w2rv"
May 25 14:03:31.735015 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:31.727826    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0f731cec-0664-4e49-891b-53bd4603dbba-dbus\") pod \"global-pull-secret-syncer-6w2rv\" (UID: \"0f731cec-0664-4e49-891b-53bd4603dbba\") " pod="kube-system/global-pull-secret-syncer-6w2rv"
May 25 14:03:31.735015 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:31.730963    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f731cec-0664-4e49-891b-53bd4603dbba-original-pull-secret\") pod \"global-pull-secret-syncer-6w2rv\" (UID: \"0f731cec-0664-4e49-891b-53bd4603dbba\") " pod="kube-system/global-pull-secret-syncer-6w2rv"
May 25 14:03:31.791078 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:31.791036    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w2rv"
May 25 14:03:31.923581 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:31.923556    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6w2rv"]
May 25 14:03:31.926447 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:03:31.926421    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f731cec_0664_4e49_891b_53bd4603dbba.slice/crio-c4a511557913e96bca8df6a49eb6e37b72903afb5163696b7b6bc165adb566e0 WatchSource:0}: Error finding container c4a511557913e96bca8df6a49eb6e37b72903afb5163696b7b6bc165adb566e0: Status 404 returned error can't find the container with id c4a511557913e96bca8df6a49eb6e37b72903afb5163696b7b6bc165adb566e0
May 25 14:03:32.655552 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:32.655512    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6w2rv" event={"ID":"0f731cec-0664-4e49-891b-53bd4603dbba","Type":"ContainerStarted","Data":"c4a511557913e96bca8df6a49eb6e37b72903afb5163696b7b6bc165adb566e0"}
May 25 14:03:35.665492 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:35.665458    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6w2rv" event={"ID":"0f731cec-0664-4e49-891b-53bd4603dbba","Type":"ContainerStarted","Data":"4cda235289cb6885475d6e63e6ef13cd18754bfa01f9b2b31b54f2b87123c440"}
May 25 14:03:35.706494 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:03:35.706439    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6w2rv" podStartSLOduration=1.141961468 podStartE2EDuration="4.706422802s" podCreationTimestamp="2026-05-25 14:03:31 +0000 UTC" firstStartedPulling="2026-05-25 14:03:31.928350404 +0000 UTC m=+219.763206475" lastFinishedPulling="2026-05-25 14:03:35.492811738 +0000 UTC m=+223.327667809" observedRunningTime="2026-05-25 14:03:35.705066646 +0000 UTC m=+223.539922739" watchObservedRunningTime="2026-05-25 14:03:35.706422802 +0000 UTC m=+223.541278895"
May 25 14:04:52.705047 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:04:52.705020    2571 kubelet.go:1628] "Image garbage collection succeeded"
May 25 14:05:38.984191 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:38.984153    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"]
May 25 14:05:38.987203 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:38.987179    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:05:38.989312 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:38.989289    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 25 14:05:38.989445 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:38.989293    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-tjw5j\""
May 25 14:05:38.989445 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:38.989298    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 25 14:05:38.989554 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:38.989303    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 25 14:05:38.989554 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:38.989319    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 25 14:05:38.989744 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:38.989731    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\""
May 25 14:05:38.989793 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:38.989742    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 25 14:05:38.989793 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:38.989753    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 25 14:05:39.002194 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:39.002161    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"]
May 25 14:05:39.063675 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:39.063643    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-online-tls\") pod \"feast-simple-feast-setup-76946f44f6-xqcj5\" (UID: \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:05:39.063847 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:39.063684    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6d9f\" (UniqueName: \"kubernetes.io/projected/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-kube-api-access-c6d9f\") pod \"feast-simple-feast-setup-76946f44f6-xqcj5\" (UID: \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:05:39.063847 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:39.063708    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-feast-data\") pod \"feast-simple-feast-setup-76946f44f6-xqcj5\" (UID: \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:05:39.063847 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:39.063762    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-offline-tls\") pod \"feast-simple-feast-setup-76946f44f6-xqcj5\" (UID: \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:05:39.063847 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:39.063792    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-ui-tls\") pod \"feast-simple-feast-setup-76946f44f6-xqcj5\" (UID: \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:05:39.064017 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:39.063872    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-registry-tls\") pod \"feast-simple-feast-setup-76946f44f6-xqcj5\" (UID: \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:05:39.164686 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:39.164652    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-registry-tls\") pod \"feast-simple-feast-setup-76946f44f6-xqcj5\" (UID: \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:05:39.164846 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:39.164700    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-online-tls\") pod \"feast-simple-feast-setup-76946f44f6-xqcj5\" (UID: \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:05:39.164846 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:39.164827    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c6d9f\" (UniqueName: \"kubernetes.io/projected/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-kube-api-access-c6d9f\") pod \"feast-simple-feast-setup-76946f44f6-xqcj5\" (UID: \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:05:39.164974 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:39.164872    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-feast-data\") pod \"feast-simple-feast-setup-76946f44f6-xqcj5\" (UID: \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:05:39.164974 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:39.164915    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-offline-tls\") pod \"feast-simple-feast-setup-76946f44f6-xqcj5\" (UID: \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:05:39.165082 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:39.164991    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-ui-tls\") pod \"feast-simple-feast-setup-76946f44f6-xqcj5\" (UID: \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:05:39.165129 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:05:39.165093    2571 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found
May 25 14:05:39.165206 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:05:39.165179    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-offline-tls podName:500e2a66-d24b-42dd-b7e6-5b6a08ba02a5 nodeName:}" failed. No retries permitted until 2026-05-25 14:05:39.665157101 +0000 UTC m=+347.500013186 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-offline-tls") pod "feast-simple-feast-setup-76946f44f6-xqcj5" (UID: "500e2a66-d24b-42dd-b7e6-5b6a08ba02a5") : secret "feast-simple-feast-setup-offline-tls" not found
May 25 14:05:39.165283 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:39.165268    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-feast-data\") pod \"feast-simple-feast-setup-76946f44f6-xqcj5\" (UID: \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:05:39.167350 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:39.167321    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-registry-tls\") pod \"feast-simple-feast-setup-76946f44f6-xqcj5\" (UID: \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:05:39.167472 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:39.167376    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-ui-tls\") pod \"feast-simple-feast-setup-76946f44f6-xqcj5\" (UID: \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:05:39.167472 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:39.167388    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-online-tls\") pod \"feast-simple-feast-setup-76946f44f6-xqcj5\" (UID: \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:05:39.172948 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:39.172910    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6d9f\" (UniqueName: \"kubernetes.io/projected/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-kube-api-access-c6d9f\") pod \"feast-simple-feast-setup-76946f44f6-xqcj5\" (UID: \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:05:39.669806 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:39.669762    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-offline-tls\") pod \"feast-simple-feast-setup-76946f44f6-xqcj5\" (UID: \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:05:39.672844 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:39.672822    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-offline-tls\") pod \"feast-simple-feast-setup-76946f44f6-xqcj5\" (UID: \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:05:39.907755 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:39.907710    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:05:40.048118 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:40.048092    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"]
May 25 14:05:40.050230 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:05:40.050203    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod500e2a66_d24b_42dd_b7e6_5b6a08ba02a5.slice/crio-3afc13c617b6e703a7b783e6c9e0b640783d66e9cefbcac5874d1bc4414b7a8f WatchSource:0}: Error finding container 3afc13c617b6e703a7b783e6c9e0b640783d66e9cefbcac5874d1bc4414b7a8f: Status 404 returned error can't find the container with id 3afc13c617b6e703a7b783e6c9e0b640783d66e9cefbcac5874d1bc4414b7a8f
May 25 14:05:40.051879 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:40.051863    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 25 14:05:41.047153 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:05:41.047061    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" event={"ID":"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5","Type":"ContainerStarted","Data":"3afc13c617b6e703a7b783e6c9e0b640783d66e9cefbcac5874d1bc4414b7a8f"}
May 25 14:06:08.138547 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:08.138509    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" event={"ID":"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5","Type":"ContainerStarted","Data":"f674a510307355ca1e6a87686fbe5824f49c03412746928042871f3263e8df0e"}
May 25 14:06:13.155532 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:13.155496    2571 generic.go:358] "Generic (PLEG): container finished" podID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerID="f674a510307355ca1e6a87686fbe5824f49c03412746928042871f3263e8df0e" exitCode=0
May 25 14:06:13.155910 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:13.155542    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" event={"ID":"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5","Type":"ContainerDied","Data":"f674a510307355ca1e6a87686fbe5824f49c03412746928042871f3263e8df0e"}
May 25 14:06:14.160538 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:14.160504    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" event={"ID":"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5","Type":"ContainerStarted","Data":"1b42d94fb7d88afb8e365e2510dd5371251a021ede170dc027cd0c7262150553"}
May 25 14:06:19.176899 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:19.176809    2571 generic.go:358] "Generic (PLEG): container finished" podID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerID="1b42d94fb7d88afb8e365e2510dd5371251a021ede170dc027cd0c7262150553" exitCode=0
May 25 14:06:19.176899 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:19.176855    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" event={"ID":"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5","Type":"ContainerDied","Data":"1b42d94fb7d88afb8e365e2510dd5371251a021ede170dc027cd0c7262150553"}
May 25 14:06:19.187552 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:19.187530    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\""
May 25 14:06:20.185019 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:20.184980    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" event={"ID":"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5","Type":"ContainerStarted","Data":"97df7127d0c1eb97409b849d3646cb4a5d88480caf8f2843eb17537c2bc8846c"}
May 25 14:06:20.185478 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:20.185033    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" event={"ID":"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5","Type":"ContainerStarted","Data":"7bde5a693fa82ed1814d924a3eca1f2dc4880e7a16fb4958fc7a8369091fd833"}
May 25 14:06:20.185478 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:20.185049    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" event={"ID":"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5","Type":"ContainerStarted","Data":"f6d4765b345b3e25622144e9c1e204d132e22c0c933b5be2d46a610cf4a72250"}
May 25 14:06:20.185478 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:20.185061    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" event={"ID":"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5","Type":"ContainerStarted","Data":"5e9d01d37d8d67e518023de18b5fa64a84f4b9e8c34cd68222064f4993082928"}
May 25 14:06:20.210467 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:20.210401    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" podStartSLOduration=14.685221489 podStartE2EDuration="42.210378709s" podCreationTimestamp="2026-05-25 14:05:38 +0000 UTC" firstStartedPulling="2026-05-25 14:05:40.052015206 +0000 UTC m=+347.886871277" lastFinishedPulling="2026-05-25 14:06:07.577172423 +0000 UTC m=+375.412028497" observedRunningTime="2026-05-25 14:06:20.208618678 +0000 UTC m=+388.043474768" watchObservedRunningTime="2026-05-25 14:06:20.210378709 +0000 UTC m=+388.045234805"
May 25 14:06:21.908820 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:21.908755    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:06:21.908820 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:21.908813    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:06:21.908820 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:21.908827    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:06:21.909425 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:21.908838    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:06:21.911030 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:21.910977    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.24:8816: connect: connection refused"
May 25 14:06:21.911204 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:21.910973    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.24:6571: connect: connection refused"
May 25 14:06:21.911204 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:21.911007    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused"
May 25 14:06:21.911325 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:21.911304    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused"
May 25 14:06:24.908633 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:24.908556    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.24:6571: connect: connection refused"
May 25 14:06:24.909114 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:24.908571    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused"
May 25 14:06:24.909114 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:24.908565    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused"
May 25 14:06:24.909114 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:24.908565    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.24:8816: connect: connection refused"
May 25 14:06:27.908733 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:27.908703    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:06:27.909213 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:27.909079    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:06:27.909304 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:27.909251    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:06:27.909512 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:27.909491    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:06:27.909578 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:27.909522    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:06:27.909614 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:27.909582    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:06:27.909614 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:27.909611    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:06:27.913593 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:27.913577    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:06:28.217909 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:28.217876    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:06:28.217909 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:28.217917    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:06:28.218477 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:28.218457    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:06:28.220778 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:06:28.220755    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:07:30.291566 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:30.291473    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"]
May 25 14:07:30.292157 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:30.291817    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="registry" containerID="cri-o://5e9d01d37d8d67e518023de18b5fa64a84f4b9e8c34cd68222064f4993082928" gracePeriod=30
May 25 14:07:30.292157 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:30.291871    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="ui" containerID="cri-o://97df7127d0c1eb97409b849d3646cb4a5d88480caf8f2843eb17537c2bc8846c" gracePeriod=30
May 25 14:07:30.292157 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:30.291955    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="online" containerID="cri-o://f6d4765b345b3e25622144e9c1e204d132e22c0c933b5be2d46a610cf4a72250" gracePeriod=30
May 25 14:07:30.292157 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:30.291874    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="offline" containerID="cri-o://7bde5a693fa82ed1814d924a3eca1f2dc4880e7a16fb4958fc7a8369091fd833" gracePeriod=30
May 25 14:07:30.565206 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:30.565118    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"]
May 25 14:07:30.569747 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:30.569726    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:30.571864 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:30.571839    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-hw6bn\""
May 25 14:07:30.582587 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:30.582556    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"]
May 25 14:07:30.706730 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:30.706697    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-registry-tls\") pod \"feast-simple-feast-setup-76946f44f6-hbnkt\" (UID: \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:30.706925 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:30.706741    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-feast-data\") pod \"feast-simple-feast-setup-76946f44f6-hbnkt\" (UID: \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:30.706925 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:30.706799    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgjm6\" (UniqueName: \"kubernetes.io/projected/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-kube-api-access-kgjm6\") pod \"feast-simple-feast-setup-76946f44f6-hbnkt\" (UID: \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:30.706925 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:30.706829    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-ui-tls\") pod \"feast-simple-feast-setup-76946f44f6-hbnkt\" (UID: \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:30.706925 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:30.706855    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-online-tls\") pod \"feast-simple-feast-setup-76946f44f6-hbnkt\" (UID: \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:30.706925 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:30.706915    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-offline-tls\") pod \"feast-simple-feast-setup-76946f44f6-hbnkt\" (UID: \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:30.807607 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:30.807561    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-registry-tls\") pod \"feast-simple-feast-setup-76946f44f6-hbnkt\" (UID: \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:30.807822 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:30.807639    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-feast-data\") pod \"feast-simple-feast-setup-76946f44f6-hbnkt\" (UID: \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:30.807822 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:30.807686    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kgjm6\" (UniqueName: \"kubernetes.io/projected/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-kube-api-access-kgjm6\") pod \"feast-simple-feast-setup-76946f44f6-hbnkt\" (UID: \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:30.807822 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:30.807723    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-ui-tls\") pod \"feast-simple-feast-setup-76946f44f6-hbnkt\" (UID: \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:30.807822 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:07:30.807744    2571 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found
May 25 14:07:30.807822 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:07:30.807822    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-registry-tls podName:3cf1c893-f7c2-40c6-a7bc-399e55a5a678 nodeName:}" failed. No retries permitted until 2026-05-25 14:07:31.307800905 +0000 UTC m=+459.142656979 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-registry-tls") pod "feast-simple-feast-setup-76946f44f6-hbnkt" (UID: "3cf1c893-f7c2-40c6-a7bc-399e55a5a678") : secret "feast-simple-feast-setup-registry-tls" not found
May 25 14:07:30.808107 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:30.807971    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-online-tls\") pod \"feast-simple-feast-setup-76946f44f6-hbnkt\" (UID: \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:30.808107 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:30.808066    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-offline-tls\") pod \"feast-simple-feast-setup-76946f44f6-hbnkt\" (UID: \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:30.808198 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:30.808135    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-feast-data\") pod \"feast-simple-feast-setup-76946f44f6-hbnkt\" (UID: \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:30.810598 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:30.810573    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-offline-tls\") pod \"feast-simple-feast-setup-76946f44f6-hbnkt\" (UID: \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:30.810887 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:30.810867    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-ui-tls\") pod \"feast-simple-feast-setup-76946f44f6-hbnkt\" (UID: \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:30.811053 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:30.811032    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-online-tls\") pod \"feast-simple-feast-setup-76946f44f6-hbnkt\" (UID: \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:30.818870 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:30.818811    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgjm6\" (UniqueName: \"kubernetes.io/projected/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-kube-api-access-kgjm6\") pod \"feast-simple-feast-setup-76946f44f6-hbnkt\" (UID: \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:31.313818 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:31.313776    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-registry-tls\") pod \"feast-simple-feast-setup-76946f44f6-hbnkt\" (UID: \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:31.316278 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:31.316258    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-registry-tls\") pod \"feast-simple-feast-setup-76946f44f6-hbnkt\" (UID: \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\") " pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:31.410980 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:31.410947    2571 generic.go:358] "Generic (PLEG): container finished" podID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerID="97df7127d0c1eb97409b849d3646cb4a5d88480caf8f2843eb17537c2bc8846c" exitCode=0
May 25 14:07:31.410980 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:31.410974    2571 generic.go:358] "Generic (PLEG): container finished" podID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerID="f6d4765b345b3e25622144e9c1e204d132e22c0c933b5be2d46a610cf4a72250" exitCode=0
May 25 14:07:31.411188 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:31.411007    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" event={"ID":"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5","Type":"ContainerDied","Data":"97df7127d0c1eb97409b849d3646cb4a5d88480caf8f2843eb17537c2bc8846c"}
May 25 14:07:31.411188 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:31.411037    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" event={"ID":"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5","Type":"ContainerDied","Data":"f6d4765b345b3e25622144e9c1e204d132e22c0c933b5be2d46a610cf4a72250"}
May 25 14:07:31.480553 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:31.480523    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:31.618663 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:31.618637    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"]
May 25 14:07:31.620587 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:07:31.620562    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3cf1c893_f7c2_40c6_a7bc_399e55a5a678.slice/crio-bb7e751b0c134fa433ffea45e1fa24ba7bf257182dbf423f72cfb7f1938513b6 WatchSource:0}: Error finding container bb7e751b0c134fa433ffea45e1fa24ba7bf257182dbf423f72cfb7f1938513b6: Status 404 returned error can't find the container with id bb7e751b0c134fa433ffea45e1fa24ba7bf257182dbf423f72cfb7f1938513b6
May 25 14:07:32.415385 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:32.415332    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" event={"ID":"3cf1c893-f7c2-40c6-a7bc-399e55a5a678","Type":"ContainerStarted","Data":"0d0fbbd2d3d4bf5ab47c26a97d8c2c30ce98c427ed6f08149dbe3f46493bd8e7"}
May 25 14:07:32.415385 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:32.415381    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" event={"ID":"3cf1c893-f7c2-40c6-a7bc-399e55a5a678","Type":"ContainerStarted","Data":"bb7e751b0c134fa433ffea45e1fa24ba7bf257182dbf423f72cfb7f1938513b6"}
May 25 14:07:36.429483 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:36.429439    2571 generic.go:358] "Generic (PLEG): container finished" podID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerID="0d0fbbd2d3d4bf5ab47c26a97d8c2c30ce98c427ed6f08149dbe3f46493bd8e7" exitCode=0
May 25 14:07:36.429818 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:36.429510    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" event={"ID":"3cf1c893-f7c2-40c6-a7bc-399e55a5a678","Type":"ContainerDied","Data":"0d0fbbd2d3d4bf5ab47c26a97d8c2c30ce98c427ed6f08149dbe3f46493bd8e7"}
May 25 14:07:37.433762 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:37.433734    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" event={"ID":"3cf1c893-f7c2-40c6-a7bc-399e55a5a678","Type":"ContainerStarted","Data":"29adfd909822aa860b675b27e67960b655075f2589a3a39b51d90b9c51124caa"}
May 25 14:07:38.218220 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:38.218175    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused"
May 25 14:07:38.218403 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:38.218191    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused"
May 25 14:07:42.451926 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:42.451891    2571 generic.go:358] "Generic (PLEG): container finished" podID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerID="29adfd909822aa860b675b27e67960b655075f2589a3a39b51d90b9c51124caa" exitCode=0
May 25 14:07:42.451926 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:42.451952    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" event={"ID":"3cf1c893-f7c2-40c6-a7bc-399e55a5a678","Type":"ContainerDied","Data":"29adfd909822aa860b675b27e67960b655075f2589a3a39b51d90b9c51124caa"}
May 25 14:07:43.459750 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:43.459702    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" event={"ID":"3cf1c893-f7c2-40c6-a7bc-399e55a5a678","Type":"ContainerStarted","Data":"b993deb7c01ddd961c53673087bf8a04abb76696891d5a161b261cd9bf6f5ed7"}
May 25 14:07:43.460260 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:43.459758    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" event={"ID":"3cf1c893-f7c2-40c6-a7bc-399e55a5a678","Type":"ContainerStarted","Data":"3e689b8e764350cf57da20e29f64be08ae5a91e1f95cfd1b04ad4bdb5803bc74"}
May 25 14:07:43.460260 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:43.459772    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" event={"ID":"3cf1c893-f7c2-40c6-a7bc-399e55a5a678","Type":"ContainerStarted","Data":"11e8f33df78cb51000874637d5dbb65cf93e1e84012e647f5cbf5528fd14419f"}
May 25 14:07:43.460260 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:43.459784    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" event={"ID":"3cf1c893-f7c2-40c6-a7bc-399e55a5a678","Type":"ContainerStarted","Data":"58a48e7fe14d2c59c4cfa031be45630cd53e64b29add84b7f7f3c26b50609e92"}
May 25 14:07:43.481126 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:43.481086    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:43.481394 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:43.481375    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:43.481526 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:43.481505    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:43.481641 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:43.481531    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:43.483887 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:43.483574    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.25:8816: connect: connection refused"
May 25 14:07:43.483887 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:43.483598    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused"
May 25 14:07:43.483887 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:43.483699    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.25:6571: connect: connection refused"
May 25 14:07:43.483887 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:43.483750    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused"
May 25 14:07:43.487002 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:43.486951    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" podStartSLOduration=13.486914724 podStartE2EDuration="13.486914724s" podCreationTimestamp="2026-05-25 14:07:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 14:07:43.484340332 +0000 UTC m=+471.319196456" watchObservedRunningTime="2026-05-25 14:07:43.486914724 +0000 UTC m=+471.321770819"
May 25 14:07:46.481453 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:46.481396    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.25:8816: connect: connection refused"
May 25 14:07:46.481453 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:46.481437    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused"
May 25 14:07:46.482010 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:46.481396    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.25:6571: connect: connection refused"
May 25 14:07:46.482010 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:46.481616    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused"
May 25 14:07:48.218832 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:48.218780    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused"
May 25 14:07:48.219344 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:48.218803    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused"
May 25 14:07:49.481523 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:49.481493    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:49.481988 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:49.481894    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:49.482205 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:49.482184    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:49.482300 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:49.482218    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:49.482509 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:49.482490    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:49.486105 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:49.486081    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:50.486990 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:50.486954    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:50.486990 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:50.486996    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:50.487435 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:50.487008    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:50.487605 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:50.487588    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:50.489893 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:50.489876    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:51.489904 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:51.489872    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:07:58.218208 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:58.218177    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused"
May 25 14:07:58.218629 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:58.218165    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused"
May 25 14:07:58.218629 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:58.218299    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:07:58.218629 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:07:58.218389    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:08:00.518429 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:00.518399    2571 generic.go:358] "Generic (PLEG): container finished" podID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerID="7bde5a693fa82ed1814d924a3eca1f2dc4880e7a16fb4958fc7a8369091fd833" exitCode=137
May 25 14:08:00.518429 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:00.518425    2571 generic.go:358] "Generic (PLEG): container finished" podID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerID="5e9d01d37d8d67e518023de18b5fa64a84f4b9e8c34cd68222064f4993082928" exitCode=137
May 25 14:08:00.518817 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:00.518477    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" event={"ID":"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5","Type":"ContainerDied","Data":"7bde5a693fa82ed1814d924a3eca1f2dc4880e7a16fb4958fc7a8369091fd833"}
May 25 14:08:00.518817 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:00.518515    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" event={"ID":"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5","Type":"ContainerDied","Data":"5e9d01d37d8d67e518023de18b5fa64a84f4b9e8c34cd68222064f4993082928"}
May 25 14:08:00.934010 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:00.933987    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:08:00.985909 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:00.985874    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-ui-tls\") pod \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\" (UID: \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\") "
May 25 14:08:00.986101 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:00.985951    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-online-tls\") pod \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\" (UID: \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\") "
May 25 14:08:00.986101 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:00.985968    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-registry-tls\") pod \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\" (UID: \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\") "
May 25 14:08:00.986101 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:00.985997    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-feast-data\") pod \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\" (UID: \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\") "
May 25 14:08:00.986101 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:00.986013    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-offline-tls\") pod \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\" (UID: \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\") "
May 25 14:08:00.986101 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:00.986043    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6d9f\" (UniqueName: \"kubernetes.io/projected/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-kube-api-access-c6d9f\") pod \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\" (UID: \"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5\") "
May 25 14:08:00.986602 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:00.986570    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-feast-data" (OuterVolumeSpecName: "feast-data") pod "500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" (UID: "500e2a66-d24b-42dd-b7e6-5b6a08ba02a5"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 14:08:00.988370 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:00.988343    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-kube-api-access-c6d9f" (OuterVolumeSpecName: "kube-api-access-c6d9f") pod "500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" (UID: "500e2a66-d24b-42dd-b7e6-5b6a08ba02a5"). InnerVolumeSpecName "kube-api-access-c6d9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 14:08:00.988468 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:00.988406    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" (UID: "500e2a66-d24b-42dd-b7e6-5b6a08ba02a5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:08:00.988710 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:00.988690    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-online-tls" (OuterVolumeSpecName: "online-tls") pod "500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" (UID: "500e2a66-d24b-42dd-b7e6-5b6a08ba02a5"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:08:00.988758 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:00.988738    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" (UID: "500e2a66-d24b-42dd-b7e6-5b6a08ba02a5"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:08:00.988795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:00.988766    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" (UID: "500e2a66-d24b-42dd-b7e6-5b6a08ba02a5"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:08:01.087403 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:01.087306    2571 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-ui-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:08:01.087403 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:01.087339    2571 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-online-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:08:01.087403 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:01.087354    2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-registry-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:08:01.087403 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:01.087366    2571 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-feast-data\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:08:01.087403 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:01.087377    2571 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-offline-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:08:01.087403 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:01.087389    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6d9f\" (UniqueName: \"kubernetes.io/projected/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5-kube-api-access-c6d9f\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:08:01.523488 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:01.523448    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5" event={"ID":"500e2a66-d24b-42dd-b7e6-5b6a08ba02a5","Type":"ContainerDied","Data":"3afc13c617b6e703a7b783e6c9e0b640783d66e9cefbcac5874d1bc4414b7a8f"}
May 25 14:08:01.523488 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:01.523489    2571 scope.go:117] "RemoveContainer" containerID="97df7127d0c1eb97409b849d3646cb4a5d88480caf8f2843eb17537c2bc8846c"
May 25 14:08:01.523999 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:01.523496    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"
May 25 14:08:01.532150 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:01.532133    2571 scope.go:117] "RemoveContainer" containerID="7bde5a693fa82ed1814d924a3eca1f2dc4880e7a16fb4958fc7a8369091fd833"
May 25 14:08:01.539986 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:01.539964    2571 scope.go:117] "RemoveContainer" containerID="f6d4765b345b3e25622144e9c1e204d132e22c0c933b5be2d46a610cf4a72250"
May 25 14:08:01.544658 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:01.544638    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"]
May 25 14:08:01.548008 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:01.547992    2571 scope.go:117] "RemoveContainer" containerID="5e9d01d37d8d67e518023de18b5fa64a84f4b9e8c34cd68222064f4993082928"
May 25 14:08:01.548173 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:01.548154    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-76946f44f6-xqcj5"]
May 25 14:08:01.555206 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:01.555187    2571 scope.go:117] "RemoveContainer" containerID="1b42d94fb7d88afb8e365e2510dd5371251a021ede170dc027cd0c7262150553"
May 25 14:08:01.562149 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:01.562128    2571 scope.go:117] "RemoveContainer" containerID="f674a510307355ca1e6a87686fbe5824f49c03412746928042871f3263e8df0e"
May 25 14:08:02.827050 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:02.827008    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" path="/var/lib/kubelet/pods/500e2a66-d24b-42dd-b7e6-5b6a08ba02a5/volumes"
May 25 14:08:54.540773 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.540741    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"]
May 25 14:08:54.541276 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.541259    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="registry"
May 25 14:08:54.541351 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.541278    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="registry"
May 25 14:08:54.541351 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.541294    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="online"
May 25 14:08:54.541351 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.541303    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="online"
May 25 14:08:54.541351 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.541316    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="ui"
May 25 14:08:54.541351 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.541325    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="ui"
May 25 14:08:54.541351 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.541338    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="feast-apply"
May 25 14:08:54.541351 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.541346    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="feast-apply"
May 25 14:08:54.541674 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.541367    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="offline"
May 25 14:08:54.541674 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.541376    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="offline"
May 25 14:08:54.541674 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.541395    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="feast-init"
May 25 14:08:54.541674 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.541403    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="feast-init"
May 25 14:08:54.541674 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.541484    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="registry"
May 25 14:08:54.541674 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.541497    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="ui"
May 25 14:08:54.541674 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.541511    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="online"
May 25 14:08:54.541674 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.541521    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="500e2a66-d24b-42dd-b7e6-5b6a08ba02a5" containerName="offline"
May 25 14:08:54.546243 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.546220    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:08:54.548284 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.548255    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 25 14:08:54.548466 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.548306    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 25 14:08:54.548466 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.548349    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\""
May 25 14:08:54.548634 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.548593    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 25 14:08:54.548634 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.548629    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 25 14:08:54.548736 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.548666    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 25 14:08:54.548736 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.548702    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-gk5tz\""
May 25 14:08:54.548919 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.548906    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 25 14:08:54.556332 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.556299    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"]
May 25 14:08:54.667612 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.667579    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-feast-data\") pod \"feast-simple-feast-remote-setup-566c978758-b7tjx\" (UID: \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:08:54.667612 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.667616    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-online-tls\") pod \"feast-simple-feast-remote-setup-566c978758-b7tjx\" (UID: \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:08:54.667859 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.667638    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-ui-tls\") pod \"feast-simple-feast-remote-setup-566c978758-b7tjx\" (UID: \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:08:54.667859 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.667666    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-offline-tls\") pod \"feast-simple-feast-remote-setup-566c978758-b7tjx\" (UID: \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:08:54.667859 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.667746    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-registry-tls\") pod \"feast-simple-feast-remote-setup-566c978758-b7tjx\" (UID: \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:08:54.667859 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.667775    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2d6x\" (UniqueName: \"kubernetes.io/projected/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-kube-api-access-j2d6x\") pod \"feast-simple-feast-remote-setup-566c978758-b7tjx\" (UID: \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:08:54.769105 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.769067    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-online-tls\") pod \"feast-simple-feast-remote-setup-566c978758-b7tjx\" (UID: \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:08:54.769296 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.769114    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-ui-tls\") pod \"feast-simple-feast-remote-setup-566c978758-b7tjx\" (UID: \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:08:54.769296 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.769157    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-offline-tls\") pod \"feast-simple-feast-remote-setup-566c978758-b7tjx\" (UID: \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:08:54.769296 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.769205    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-registry-tls\") pod \"feast-simple-feast-remote-setup-566c978758-b7tjx\" (UID: \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:08:54.769296 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:08:54.769219    2571 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found
May 25 14:08:54.769296 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:08:54.769295    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-online-tls podName:2d39ac07-48ea-4c53-a62d-cdb2b205cfc7 nodeName:}" failed. No retries permitted until 2026-05-25 14:08:55.269270961 +0000 UTC m=+543.104127222 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-online-tls") pod "feast-simple-feast-remote-setup-566c978758-b7tjx" (UID: "2d39ac07-48ea-4c53-a62d-cdb2b205cfc7") : secret "feast-simple-feast-remote-setup-online-tls" not found
May 25 14:08:54.769555 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.769319    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j2d6x\" (UniqueName: \"kubernetes.io/projected/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-kube-api-access-j2d6x\") pod \"feast-simple-feast-remote-setup-566c978758-b7tjx\" (UID: \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:08:54.769555 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.769407    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-feast-data\") pod \"feast-simple-feast-remote-setup-566c978758-b7tjx\" (UID: \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:08:54.769756 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.769734    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-feast-data\") pod \"feast-simple-feast-remote-setup-566c978758-b7tjx\" (UID: \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:08:54.769971 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.769926    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-registry-tls\") pod \"feast-simple-feast-remote-setup-566c978758-b7tjx\" (UID: \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:08:54.771782 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.771763    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-ui-tls\") pod \"feast-simple-feast-remote-setup-566c978758-b7tjx\" (UID: \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:08:54.771782 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.771775    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-offline-tls\") pod \"feast-simple-feast-remote-setup-566c978758-b7tjx\" (UID: \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:08:54.778136 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:54.778113    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2d6x\" (UniqueName: \"kubernetes.io/projected/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-kube-api-access-j2d6x\") pod \"feast-simple-feast-remote-setup-566c978758-b7tjx\" (UID: \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:08:55.274286 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:55.274249    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-online-tls\") pod \"feast-simple-feast-remote-setup-566c978758-b7tjx\" (UID: \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:08:55.276691 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:55.276673    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-online-tls\") pod \"feast-simple-feast-remote-setup-566c978758-b7tjx\" (UID: \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:08:55.457054 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:55.457018    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:08:55.594913 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:55.594890    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"]
May 25 14:08:55.597607 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:08:55.597577    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d39ac07_48ea_4c53_a62d_cdb2b205cfc7.slice/crio-b04d12f291cace6b4af90c0ead699bd9b3fac80131d913b51399aa47e7a70c10 WatchSource:0}: Error finding container b04d12f291cace6b4af90c0ead699bd9b3fac80131d913b51399aa47e7a70c10: Status 404 returned error can't find the container with id b04d12f291cace6b4af90c0ead699bd9b3fac80131d913b51399aa47e7a70c10
May 25 14:08:55.688408 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:55.688379    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx" event={"ID":"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7","Type":"ContainerStarted","Data":"a6982a56513cdae9078b2589ad7bd503f2a97884fba5d671f265b994aed2d740"}
May 25 14:08:55.688408 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:08:55.688413    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx" event={"ID":"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7","Type":"ContainerStarted","Data":"b04d12f291cace6b4af90c0ead699bd9b3fac80131d913b51399aa47e7a70c10"}
May 25 14:09:00.704660 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:09:00.704618    2571 generic.go:358] "Generic (PLEG): container finished" podID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerID="a6982a56513cdae9078b2589ad7bd503f2a97884fba5d671f265b994aed2d740" exitCode=0
May 25 14:09:00.705127 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:09:00.704695    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx" event={"ID":"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7","Type":"ContainerDied","Data":"a6982a56513cdae9078b2589ad7bd503f2a97884fba5d671f265b994aed2d740"}
May 25 14:09:01.709389 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:09:01.709354    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx" event={"ID":"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7","Type":"ContainerStarted","Data":"5db31ec9545f983afdf84d91955d4417ec653fcab2d675deec06becbf2f2a8b8"}
May 25 14:09:06.726561 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:09:06.726526    2571 generic.go:358] "Generic (PLEG): container finished" podID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerID="5db31ec9545f983afdf84d91955d4417ec653fcab2d675deec06becbf2f2a8b8" exitCode=0
May 25 14:09:06.727066 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:09:06.726581    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx" event={"ID":"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7","Type":"ContainerDied","Data":"5db31ec9545f983afdf84d91955d4417ec653fcab2d675deec06becbf2f2a8b8"}
May 25 14:09:07.733984 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:09:07.733923    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx" event={"ID":"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7","Type":"ContainerStarted","Data":"c4b891f92f249554c423c14f1742b17fad0dcb829c6a2425a6766e50bee6dd5c"}
May 25 14:09:07.733984 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:09:07.733992    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx" event={"ID":"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7","Type":"ContainerStarted","Data":"39916cacc1645502465224e642a0eb16ef9f2ee5a6d17eee3faf2723c25be7aa"}
May 25 14:09:07.734509 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:09:07.734004    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx" event={"ID":"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7","Type":"ContainerStarted","Data":"c9e22cc56abb1d4271676329f2e663a48583410c985b3d39740069805ae565ea"}
May 25 14:09:07.759762 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:09:07.759698    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx" podStartSLOduration=13.759677932 podStartE2EDuration="13.759677932s" podCreationTimestamp="2026-05-25 14:08:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 14:09:07.756565409 +0000 UTC m=+555.591421502" watchObservedRunningTime="2026-05-25 14:09:07.759677932 +0000 UTC m=+555.594534026"
May 25 14:09:10.457655 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:09:10.457613    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:09:10.458254 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:09:10.458226    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:09:10.458254 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:09:10.458254    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:09:10.459335 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:09:10.459303    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx" podUID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused"
May 25 14:09:10.459434 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:09:10.459323    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx" podUID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.26:8816: connect: connection refused"
May 25 14:09:10.459434 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:09:10.459348    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx" podUID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused"
May 25 14:09:13.458808 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:09:13.458778    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:09:13.459275 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:09:13.459254    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:09:13.459326 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:09:13.459290    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:09:13.459495 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:09:13.459478    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:09:13.463480 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:09:13.463462    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:09:13.753454 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:09:13.753369    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:09:13.753454 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:09:13.753436    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:09:13.753738 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:09:13.753720    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:09:13.756368 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:09:13.756350    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:10:16.346181 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:16.346087    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"]
May 25 14:10:16.346730 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:16.346442    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx" podUID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerName="online" containerID="cri-o://c9e22cc56abb1d4271676329f2e663a48583410c985b3d39740069805ae565ea" gracePeriod=30
May 25 14:10:16.346730 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:16.346599    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx" podUID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerName="offline" containerID="cri-o://39916cacc1645502465224e642a0eb16ef9f2ee5a6d17eee3faf2723c25be7aa" gracePeriod=30
May 25 14:10:16.346730 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:16.346576    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx" podUID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerName="ui" containerID="cri-o://c4b891f92f249554c423c14f1742b17fad0dcb829c6a2425a6766e50bee6dd5c" gracePeriod=30
May 25 14:10:16.517129 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:16.517099    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"]
May 25 14:10:16.517478 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:16.517449    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="registry" containerID="cri-o://58a48e7fe14d2c59c4cfa031be45630cd53e64b29add84b7f7f3c26b50609e92" gracePeriod=30
May 25 14:10:16.517594 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:16.517501    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="offline" containerID="cri-o://3e689b8e764350cf57da20e29f64be08ae5a91e1f95cfd1b04ad4bdb5803bc74" gracePeriod=30
May 25 14:10:16.517594 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:16.517534    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="online" containerID="cri-o://11e8f33df78cb51000874637d5dbb65cf93e1e84012e647f5cbf5528fd14419f" gracePeriod=30
May 25 14:10:16.517706 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:16.517486    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="ui" containerID="cri-o://b993deb7c01ddd961c53673087bf8a04abb76696891d5a161b261cd9bf6f5ed7" gracePeriod=30
May 25 14:10:17.954422 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:17.954390    2571 generic.go:358] "Generic (PLEG): container finished" podID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerID="b993deb7c01ddd961c53673087bf8a04abb76696891d5a161b261cd9bf6f5ed7" exitCode=0
May 25 14:10:17.954422 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:17.954416    2571 generic.go:358] "Generic (PLEG): container finished" podID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerID="11e8f33df78cb51000874637d5dbb65cf93e1e84012e647f5cbf5528fd14419f" exitCode=0
May 25 14:10:17.954883 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:17.954462    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" event={"ID":"3cf1c893-f7c2-40c6-a7bc-399e55a5a678","Type":"ContainerDied","Data":"b993deb7c01ddd961c53673087bf8a04abb76696891d5a161b261cd9bf6f5ed7"}
May 25 14:10:17.954883 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:17.954497    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" event={"ID":"3cf1c893-f7c2-40c6-a7bc-399e55a5a678","Type":"ContainerDied","Data":"11e8f33df78cb51000874637d5dbb65cf93e1e84012e647f5cbf5528fd14419f"}
May 25 14:10:17.956795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:17.956775    2571 generic.go:358] "Generic (PLEG): container finished" podID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerID="c9e22cc56abb1d4271676329f2e663a48583410c985b3d39740069805ae565ea" exitCode=0
May 25 14:10:17.956905 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:17.956848    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx" event={"ID":"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7","Type":"ContainerDied","Data":"c9e22cc56abb1d4271676329f2e663a48583410c985b3d39740069805ae565ea"}
May 25 14:10:19.964885 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:19.964854    2571 generic.go:358] "Generic (PLEG): container finished" podID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerID="c4b891f92f249554c423c14f1742b17fad0dcb829c6a2425a6766e50bee6dd5c" exitCode=0
May 25 14:10:19.965262 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:19.964923    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx" event={"ID":"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7","Type":"ContainerDied","Data":"c4b891f92f249554c423c14f1742b17fad0dcb829c6a2425a6766e50bee6dd5c"}
May 25 14:10:20.487322 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:20.487284    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused"
May 25 14:10:20.487322 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:20.487303    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused"
May 25 14:10:23.459782 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:23.459729    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx" podUID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused"
May 25 14:10:23.753693 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:23.753590    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx" podUID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused"
May 25 14:10:30.487326 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:30.487283    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused"
May 25 14:10:30.487326 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:30.487306    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused"
May 25 14:10:33.459926 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:33.459883    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx" podUID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused"
May 25 14:10:33.754349 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:33.754251    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx" podUID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused"
May 25 14:10:40.487415 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:40.487307    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused"
May 25 14:10:40.487415 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:40.487361    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused"
May 25 14:10:40.487828 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:40.487462    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:10:40.487828 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:40.487527    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:10:43.459565 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:43.459522    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx" podUID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused"
May 25 14:10:43.460065 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:43.459635    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:10:43.753744 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:43.753643    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx" podUID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused"
May 25 14:10:43.753914 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:43.753767    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:10:46.541399 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:46.541373    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:10:46.663582 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:46.663510    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-online-tls\") pod \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\" (UID: \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\") "
May 25 14:10:46.663582 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:46.663552    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2d6x\" (UniqueName: \"kubernetes.io/projected/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-kube-api-access-j2d6x\") pod \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\" (UID: \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\") "
May 25 14:10:46.663582 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:46.663579    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-offline-tls\") pod \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\" (UID: \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\") "
May 25 14:10:46.663816 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:46.663641    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-ui-tls\") pod \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\" (UID: \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\") "
May 25 14:10:46.663816 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:46.663672    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-feast-data\") pod \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\" (UID: \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\") "
May 25 14:10:46.663816 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:46.663686    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-registry-tls\") pod \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\" (UID: \"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7\") "
May 25 14:10:46.664141 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:46.664108    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" (UID: "2d39ac07-48ea-4c53-a62d-cdb2b205cfc7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:10:46.664261 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:46.664183    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-feast-data" (OuterVolumeSpecName: "feast-data") pod "2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" (UID: "2d39ac07-48ea-4c53-a62d-cdb2b205cfc7"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 14:10:46.665871 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:46.665845    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" (UID: "2d39ac07-48ea-4c53-a62d-cdb2b205cfc7"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:10:46.665871 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:46.665861    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-kube-api-access-j2d6x" (OuterVolumeSpecName: "kube-api-access-j2d6x") pod "2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" (UID: "2d39ac07-48ea-4c53-a62d-cdb2b205cfc7"). InnerVolumeSpecName "kube-api-access-j2d6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 14:10:46.666133 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:46.666116    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-online-tls" (OuterVolumeSpecName: "online-tls") pod "2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" (UID: "2d39ac07-48ea-4c53-a62d-cdb2b205cfc7"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:10:46.666252 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:46.666148    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" (UID: "2d39ac07-48ea-4c53-a62d-cdb2b205cfc7"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:10:46.764530 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:46.764497    2571 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-feast-data\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:10:46.764530 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:46.764522    2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-registry-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:10:46.764530 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:46.764532    2571 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-online-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:10:46.764744 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:46.764540    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2d6x\" (UniqueName: \"kubernetes.io/projected/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-kube-api-access-j2d6x\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:10:46.764744 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:46.764549    2571 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-offline-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:10:46.764744 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:46.764557    2571 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7-ui-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:10:47.047523 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.047480    2571 generic.go:358] "Generic (PLEG): container finished" podID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerID="3e689b8e764350cf57da20e29f64be08ae5a91e1f95cfd1b04ad4bdb5803bc74" exitCode=137
May 25 14:10:47.047523 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.047506    2571 generic.go:358] "Generic (PLEG): container finished" podID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerID="58a48e7fe14d2c59c4cfa031be45630cd53e64b29add84b7f7f3c26b50609e92" exitCode=137
May 25 14:10:47.047701 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.047554    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" event={"ID":"3cf1c893-f7c2-40c6-a7bc-399e55a5a678","Type":"ContainerDied","Data":"3e689b8e764350cf57da20e29f64be08ae5a91e1f95cfd1b04ad4bdb5803bc74"}
May 25 14:10:47.047701 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.047593    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" event={"ID":"3cf1c893-f7c2-40c6-a7bc-399e55a5a678","Type":"ContainerDied","Data":"58a48e7fe14d2c59c4cfa031be45630cd53e64b29add84b7f7f3c26b50609e92"}
May 25 14:10:47.049858 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.049838    2571 generic.go:358] "Generic (PLEG): container finished" podID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerID="39916cacc1645502465224e642a0eb16ef9f2ee5a6d17eee3faf2723c25be7aa" exitCode=137
May 25 14:10:47.050001 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.049960    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"
May 25 14:10:47.050051 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.049951    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx" event={"ID":"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7","Type":"ContainerDied","Data":"39916cacc1645502465224e642a0eb16ef9f2ee5a6d17eee3faf2723c25be7aa"}
May 25 14:10:47.050095 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.050067    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx" event={"ID":"2d39ac07-48ea-4c53-a62d-cdb2b205cfc7","Type":"ContainerDied","Data":"b04d12f291cace6b4af90c0ead699bd9b3fac80131d913b51399aa47e7a70c10"}
May 25 14:10:47.050095 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.050083    2571 scope.go:117] "RemoveContainer" containerID="c4b891f92f249554c423c14f1742b17fad0dcb829c6a2425a6766e50bee6dd5c"
May 25 14:10:47.060455 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.060437    2571 scope.go:117] "RemoveContainer" containerID="39916cacc1645502465224e642a0eb16ef9f2ee5a6d17eee3faf2723c25be7aa"
May 25 14:10:47.070034 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.070010    2571 scope.go:117] "RemoveContainer" containerID="c9e22cc56abb1d4271676329f2e663a48583410c985b3d39740069805ae565ea"
May 25 14:10:47.071361 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.071341    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"]
May 25 14:10:47.076984 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.076958    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-566c978758-b7tjx"]
May 25 14:10:47.078641 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.078622    2571 scope.go:117] "RemoveContainer" containerID="5db31ec9545f983afdf84d91955d4417ec653fcab2d675deec06becbf2f2a8b8"
May 25 14:10:47.086122 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.086105    2571 scope.go:117] "RemoveContainer" containerID="a6982a56513cdae9078b2589ad7bd503f2a97884fba5d671f265b994aed2d740"
May 25 14:10:47.096156 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.096134    2571 scope.go:117] "RemoveContainer" containerID="c4b891f92f249554c423c14f1742b17fad0dcb829c6a2425a6766e50bee6dd5c"
May 25 14:10:47.096463 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:10:47.096436    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4b891f92f249554c423c14f1742b17fad0dcb829c6a2425a6766e50bee6dd5c\": container with ID starting with c4b891f92f249554c423c14f1742b17fad0dcb829c6a2425a6766e50bee6dd5c not found: ID does not exist" containerID="c4b891f92f249554c423c14f1742b17fad0dcb829c6a2425a6766e50bee6dd5c"
May 25 14:10:47.096566 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.096470    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4b891f92f249554c423c14f1742b17fad0dcb829c6a2425a6766e50bee6dd5c"} err="failed to get container status \"c4b891f92f249554c423c14f1742b17fad0dcb829c6a2425a6766e50bee6dd5c\": rpc error: code = NotFound desc = could not find container \"c4b891f92f249554c423c14f1742b17fad0dcb829c6a2425a6766e50bee6dd5c\": container with ID starting with c4b891f92f249554c423c14f1742b17fad0dcb829c6a2425a6766e50bee6dd5c not found: ID does not exist"
May 25 14:10:47.096566 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.096487    2571 scope.go:117] "RemoveContainer" containerID="39916cacc1645502465224e642a0eb16ef9f2ee5a6d17eee3faf2723c25be7aa"
May 25 14:10:47.096731 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:10:47.096706    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39916cacc1645502465224e642a0eb16ef9f2ee5a6d17eee3faf2723c25be7aa\": container with ID starting with 39916cacc1645502465224e642a0eb16ef9f2ee5a6d17eee3faf2723c25be7aa not found: ID does not exist" containerID="39916cacc1645502465224e642a0eb16ef9f2ee5a6d17eee3faf2723c25be7aa"
May 25 14:10:47.096796 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.096739    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39916cacc1645502465224e642a0eb16ef9f2ee5a6d17eee3faf2723c25be7aa"} err="failed to get container status \"39916cacc1645502465224e642a0eb16ef9f2ee5a6d17eee3faf2723c25be7aa\": rpc error: code = NotFound desc = could not find container \"39916cacc1645502465224e642a0eb16ef9f2ee5a6d17eee3faf2723c25be7aa\": container with ID starting with 39916cacc1645502465224e642a0eb16ef9f2ee5a6d17eee3faf2723c25be7aa not found: ID does not exist"
May 25 14:10:47.096796 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.096764    2571 scope.go:117] "RemoveContainer" containerID="c9e22cc56abb1d4271676329f2e663a48583410c985b3d39740069805ae565ea"
May 25 14:10:47.097210 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:10:47.097187    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9e22cc56abb1d4271676329f2e663a48583410c985b3d39740069805ae565ea\": container with ID starting with c9e22cc56abb1d4271676329f2e663a48583410c985b3d39740069805ae565ea not found: ID does not exist" containerID="c9e22cc56abb1d4271676329f2e663a48583410c985b3d39740069805ae565ea"
May 25 14:10:47.097292 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.097219    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9e22cc56abb1d4271676329f2e663a48583410c985b3d39740069805ae565ea"} err="failed to get container status \"c9e22cc56abb1d4271676329f2e663a48583410c985b3d39740069805ae565ea\": rpc error: code = NotFound desc = could not find container \"c9e22cc56abb1d4271676329f2e663a48583410c985b3d39740069805ae565ea\": container with ID starting with c9e22cc56abb1d4271676329f2e663a48583410c985b3d39740069805ae565ea not found: ID does not exist"
May 25 14:10:47.097292 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.097242    2571 scope.go:117] "RemoveContainer" containerID="5db31ec9545f983afdf84d91955d4417ec653fcab2d675deec06becbf2f2a8b8"
May 25 14:10:47.097516 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:10:47.097498    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5db31ec9545f983afdf84d91955d4417ec653fcab2d675deec06becbf2f2a8b8\": container with ID starting with 5db31ec9545f983afdf84d91955d4417ec653fcab2d675deec06becbf2f2a8b8 not found: ID does not exist" containerID="5db31ec9545f983afdf84d91955d4417ec653fcab2d675deec06becbf2f2a8b8"
May 25 14:10:47.097568 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.097520    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5db31ec9545f983afdf84d91955d4417ec653fcab2d675deec06becbf2f2a8b8"} err="failed to get container status \"5db31ec9545f983afdf84d91955d4417ec653fcab2d675deec06becbf2f2a8b8\": rpc error: code = NotFound desc = could not find container \"5db31ec9545f983afdf84d91955d4417ec653fcab2d675deec06becbf2f2a8b8\": container with ID starting with 5db31ec9545f983afdf84d91955d4417ec653fcab2d675deec06becbf2f2a8b8 not found: ID does not exist"
May 25 14:10:47.097568 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.097537    2571 scope.go:117] "RemoveContainer" containerID="a6982a56513cdae9078b2589ad7bd503f2a97884fba5d671f265b994aed2d740"
May 25 14:10:47.097774 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:10:47.097758    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6982a56513cdae9078b2589ad7bd503f2a97884fba5d671f265b994aed2d740\": container with ID starting with a6982a56513cdae9078b2589ad7bd503f2a97884fba5d671f265b994aed2d740 not found: ID does not exist" containerID="a6982a56513cdae9078b2589ad7bd503f2a97884fba5d671f265b994aed2d740"
May 25 14:10:47.097823 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.097778    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6982a56513cdae9078b2589ad7bd503f2a97884fba5d671f265b994aed2d740"} err="failed to get container status \"a6982a56513cdae9078b2589ad7bd503f2a97884fba5d671f265b994aed2d740\": rpc error: code = NotFound desc = could not find container \"a6982a56513cdae9078b2589ad7bd503f2a97884fba5d671f265b994aed2d740\": container with ID starting with a6982a56513cdae9078b2589ad7bd503f2a97884fba5d671f265b994aed2d740 not found: ID does not exist"
May 25 14:10:47.164755 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.164727    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:10:47.168234 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.168213    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-registry-tls\") pod \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\" (UID: \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\") "
May 25 14:10:47.168307 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.168275    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-feast-data\") pod \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\" (UID: \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\") "
May 25 14:10:47.168345 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.168311    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-ui-tls\") pod \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\" (UID: \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\") "
May 25 14:10:47.168345 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.168337    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-online-tls\") pod \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\" (UID: \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\") "
May 25 14:10:47.168407 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.168390    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgjm6\" (UniqueName: \"kubernetes.io/projected/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-kube-api-access-kgjm6\") pod \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\" (UID: \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\") "
May 25 14:10:47.168456 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.168413    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-offline-tls\") pod \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\" (UID: \"3cf1c893-f7c2-40c6-a7bc-399e55a5a678\") "
May 25 14:10:47.168852 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.168823    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-feast-data" (OuterVolumeSpecName: "feast-data") pod "3cf1c893-f7c2-40c6-a7bc-399e55a5a678" (UID: "3cf1c893-f7c2-40c6-a7bc-399e55a5a678"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 14:10:47.170717 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.170674    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "3cf1c893-f7c2-40c6-a7bc-399e55a5a678" (UID: "3cf1c893-f7c2-40c6-a7bc-399e55a5a678"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:10:47.171001 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.170980    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "3cf1c893-f7c2-40c6-a7bc-399e55a5a678" (UID: "3cf1c893-f7c2-40c6-a7bc-399e55a5a678"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:10:47.171140 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.171103    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-online-tls" (OuterVolumeSpecName: "online-tls") pod "3cf1c893-f7c2-40c6-a7bc-399e55a5a678" (UID: "3cf1c893-f7c2-40c6-a7bc-399e55a5a678"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:10:47.171232 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.171161    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "3cf1c893-f7c2-40c6-a7bc-399e55a5a678" (UID: "3cf1c893-f7c2-40c6-a7bc-399e55a5a678"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:10:47.171279 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.171256    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-kube-api-access-kgjm6" (OuterVolumeSpecName: "kube-api-access-kgjm6") pod "3cf1c893-f7c2-40c6-a7bc-399e55a5a678" (UID: "3cf1c893-f7c2-40c6-a7bc-399e55a5a678"). InnerVolumeSpecName "kube-api-access-kgjm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 14:10:47.269682 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.269648    2571 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-feast-data\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:10:47.269682 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.269676    2571 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-ui-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:10:47.269682 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.269685    2571 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-online-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:10:47.269904 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.269694    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgjm6\" (UniqueName: \"kubernetes.io/projected/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-kube-api-access-kgjm6\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:10:47.269904 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.269705    2571 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-offline-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:10:47.269904 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:47.269713    2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3cf1c893-f7c2-40c6-a7bc-399e55a5a678-registry-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:10:48.055659 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:48.055619    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt" event={"ID":"3cf1c893-f7c2-40c6-a7bc-399e55a5a678","Type":"ContainerDied","Data":"bb7e751b0c134fa433ffea45e1fa24ba7bf257182dbf423f72cfb7f1938513b6"}
May 25 14:10:48.056117 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:48.055636    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"
May 25 14:10:48.056117 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:48.055675    2571 scope.go:117] "RemoveContainer" containerID="b993deb7c01ddd961c53673087bf8a04abb76696891d5a161b261cd9bf6f5ed7"
May 25 14:10:48.063839 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:48.063822    2571 scope.go:117] "RemoveContainer" containerID="3e689b8e764350cf57da20e29f64be08ae5a91e1f95cfd1b04ad4bdb5803bc74"
May 25 14:10:48.070826 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:48.070809    2571 scope.go:117] "RemoveContainer" containerID="11e8f33df78cb51000874637d5dbb65cf93e1e84012e647f5cbf5528fd14419f"
May 25 14:10:48.076738 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:48.076715    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"]
May 25 14:10:48.078791 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:48.078769    2571 scope.go:117] "RemoveContainer" containerID="58a48e7fe14d2c59c4cfa031be45630cd53e64b29add84b7f7f3c26b50609e92"
May 25 14:10:48.079910 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:48.079893    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-76946f44f6-hbnkt"]
May 25 14:10:48.085696 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:48.085680    2571 scope.go:117] "RemoveContainer" containerID="29adfd909822aa860b675b27e67960b655075f2589a3a39b51d90b9c51124caa"
May 25 14:10:48.092704 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:48.092687    2571 scope.go:117] "RemoveContainer" containerID="0d0fbbd2d3d4bf5ab47c26a97d8c2c30ce98c427ed6f08149dbe3f46493bd8e7"
May 25 14:10:48.825791 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:48.825760    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" path="/var/lib/kubelet/pods/2d39ac07-48ea-4c53-a62d-cdb2b205cfc7/volumes"
May 25 14:10:48.826296 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:48.826283    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" path="/var/lib/kubelet/pods/3cf1c893-f7c2-40c6-a7bc-399e55a5a678/volumes"
May 25 14:10:59.650914 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.650879    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-pt8f6"]
May 25 14:10:59.651408 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651225    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="feast-apply"
May 25 14:10:59.651408 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651238    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="feast-apply"
May 25 14:10:59.651408 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651249    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerName="feast-apply"
May 25 14:10:59.651408 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651254    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerName="feast-apply"
May 25 14:10:59.651408 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651262    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="online"
May 25 14:10:59.651408 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651271    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="online"
May 25 14:10:59.651408 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651284    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="feast-init"
May 25 14:10:59.651408 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651290    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="feast-init"
May 25 14:10:59.651408 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651296    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerName="ui"
May 25 14:10:59.651408 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651302    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerName="ui"
May 25 14:10:59.651408 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651313    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="offline"
May 25 14:10:59.651408 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651318    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="offline"
May 25 14:10:59.651408 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651324    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="ui"
May 25 14:10:59.651408 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651329    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="ui"
May 25 14:10:59.651408 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651337    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="registry"
May 25 14:10:59.651408 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651342    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="registry"
May 25 14:10:59.651408 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651354    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerName="online"
May 25 14:10:59.651408 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651362    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerName="online"
May 25 14:10:59.651408 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651369    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerName="feast-init"
May 25 14:10:59.651408 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651374    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerName="feast-init"
May 25 14:10:59.651408 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651382    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerName="offline"
May 25 14:10:59.651408 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651387    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerName="offline"
May 25 14:10:59.652258 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651433    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="registry"
May 25 14:10:59.652258 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651443    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="online"
May 25 14:10:59.652258 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651449    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerName="offline"
May 25 14:10:59.652258 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651455    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="ui"
May 25 14:10:59.652258 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651461    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerName="ui"
May 25 14:10:59.652258 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651467    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="2d39ac07-48ea-4c53-a62d-cdb2b205cfc7" containerName="online"
May 25 14:10:59.652258 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.651473    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="3cf1c893-f7c2-40c6-a7bc-399e55a5a678" containerName="offline"
May 25 14:10:59.656733 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.656714    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-pt8f6"
May 25 14:10:59.659912 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.659499    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 25 14:10:59.659912 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.659512    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\""
May 25 14:10:59.659912 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.659762    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-p94zl\""
May 25 14:10:59.660570 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.660547    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 25 14:10:59.660688 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.660610    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-pt8f6"]
May 25 14:10:59.773491 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.773454    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t482c\" (UniqueName: \"kubernetes.io/projected/7a3f8f28-e07b-4bf7-a12a-fd034d8faba6-kube-api-access-t482c\") pod \"postgres-7c7656cc6d-pt8f6\" (UID: \"7a3f8f28-e07b-4bf7-a12a-fd034d8faba6\") " pod="test-ns-feast/postgres-7c7656cc6d-pt8f6"
May 25 14:10:59.773635 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.773523    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/7a3f8f28-e07b-4bf7-a12a-fd034d8faba6-postgresdata\") pod \"postgres-7c7656cc6d-pt8f6\" (UID: \"7a3f8f28-e07b-4bf7-a12a-fd034d8faba6\") " pod="test-ns-feast/postgres-7c7656cc6d-pt8f6"
May 25 14:10:59.874742 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.874714    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/7a3f8f28-e07b-4bf7-a12a-fd034d8faba6-postgresdata\") pod \"postgres-7c7656cc6d-pt8f6\" (UID: \"7a3f8f28-e07b-4bf7-a12a-fd034d8faba6\") " pod="test-ns-feast/postgres-7c7656cc6d-pt8f6"
May 25 14:10:59.874868 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.874796    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t482c\" (UniqueName: \"kubernetes.io/projected/7a3f8f28-e07b-4bf7-a12a-fd034d8faba6-kube-api-access-t482c\") pod \"postgres-7c7656cc6d-pt8f6\" (UID: \"7a3f8f28-e07b-4bf7-a12a-fd034d8faba6\") " pod="test-ns-feast/postgres-7c7656cc6d-pt8f6"
May 25 14:10:59.875113 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.875093    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/7a3f8f28-e07b-4bf7-a12a-fd034d8faba6-postgresdata\") pod \"postgres-7c7656cc6d-pt8f6\" (UID: \"7a3f8f28-e07b-4bf7-a12a-fd034d8faba6\") " pod="test-ns-feast/postgres-7c7656cc6d-pt8f6"
May 25 14:10:59.883093 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.883067    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t482c\" (UniqueName: \"kubernetes.io/projected/7a3f8f28-e07b-4bf7-a12a-fd034d8faba6-kube-api-access-t482c\") pod \"postgres-7c7656cc6d-pt8f6\" (UID: \"7a3f8f28-e07b-4bf7-a12a-fd034d8faba6\") " pod="test-ns-feast/postgres-7c7656cc6d-pt8f6"
May 25 14:10:59.974175 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:10:59.974148    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-pt8f6"
May 25 14:11:00.097138 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:00.097103    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-pt8f6"]
May 25 14:11:00.101174 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:11:00.101151    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a3f8f28_e07b_4bf7_a12a_fd034d8faba6.slice/crio-9864b6fde48cf3bf2b577bb9dfb84cd79e39f9085238429340d6060500de0e0e WatchSource:0}: Error finding container 9864b6fde48cf3bf2b577bb9dfb84cd79e39f9085238429340d6060500de0e0e: Status 404 returned error can't find the container with id 9864b6fde48cf3bf2b577bb9dfb84cd79e39f9085238429340d6060500de0e0e
May 25 14:11:00.102781 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:00.102765    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 25 14:11:01.096575 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:01.096539    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-pt8f6" event={"ID":"7a3f8f28-e07b-4bf7-a12a-fd034d8faba6","Type":"ContainerStarted","Data":"9864b6fde48cf3bf2b577bb9dfb84cd79e39f9085238429340d6060500de0e0e"}
May 25 14:11:08.121679 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:08.121646    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-pt8f6" event={"ID":"7a3f8f28-e07b-4bf7-a12a-fd034d8faba6","Type":"ContainerStarted","Data":"65fc63beea848c38768f319ba883919a91f61b8c1da727d62f07e36d05919f20"}
May 25 14:11:08.139565 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:08.139516    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/postgres-7c7656cc6d-pt8f6" podStartSLOduration=1.5735193189999999 podStartE2EDuration="9.139499719s" podCreationTimestamp="2026-05-25 14:10:59 +0000 UTC" firstStartedPulling="2026-05-25 14:11:00.102910391 +0000 UTC m=+667.937766462" lastFinishedPulling="2026-05-25 14:11:07.668890792 +0000 UTC m=+675.503746862" observedRunningTime="2026-05-25 14:11:08.137708005 +0000 UTC m=+675.972564099" watchObservedRunningTime="2026-05-25 14:11:08.139499719 +0000 UTC m=+675.974355811"
May 25 14:11:12.510751 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:12.510715    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-9cc989665-755hx"]
May 25 14:11:12.514838 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:12.514819    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:12.517999 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:12.517978    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-9mqn4\""
May 25 14:11:12.518107 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:12.518045    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\""
May 25 14:11:12.518107 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:12.518056    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\""
May 25 14:11:12.518107 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:12.518093    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\""
May 25 14:11:12.518326 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:12.518310    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\""
May 25 14:11:12.524833 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:12.524812    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-9cc989665-755hx"]
May 25 14:11:12.591125 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:12.591100    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/36732a46-1407-4b59-9553-6968a51537c7-offline-tls\") pod \"feast-credit-scoring-9cc989665-755hx\" (UID: \"36732a46-1407-4b59-9553-6968a51537c7\") " pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:12.591267 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:12.591134    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/36732a46-1407-4b59-9553-6968a51537c7-online-tls\") pod \"feast-credit-scoring-9cc989665-755hx\" (UID: \"36732a46-1407-4b59-9553-6968a51537c7\") " pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:12.591267 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:12.591175    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/36732a46-1407-4b59-9553-6968a51537c7-registry-tls\") pod \"feast-credit-scoring-9cc989665-755hx\" (UID: \"36732a46-1407-4b59-9553-6968a51537c7\") " pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:12.591267 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:12.591194    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/36732a46-1407-4b59-9553-6968a51537c7-feast-data\") pod \"feast-credit-scoring-9cc989665-755hx\" (UID: \"36732a46-1407-4b59-9553-6968a51537c7\") " pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:12.591267 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:12.591226    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk6ql\" (UniqueName: \"kubernetes.io/projected/36732a46-1407-4b59-9553-6968a51537c7-kube-api-access-kk6ql\") pod \"feast-credit-scoring-9cc989665-755hx\" (UID: \"36732a46-1407-4b59-9553-6968a51537c7\") " pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:12.692032 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:12.691997    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kk6ql\" (UniqueName: \"kubernetes.io/projected/36732a46-1407-4b59-9553-6968a51537c7-kube-api-access-kk6ql\") pod \"feast-credit-scoring-9cc989665-755hx\" (UID: \"36732a46-1407-4b59-9553-6968a51537c7\") " pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:12.692191 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:12.692067    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/36732a46-1407-4b59-9553-6968a51537c7-offline-tls\") pod \"feast-credit-scoring-9cc989665-755hx\" (UID: \"36732a46-1407-4b59-9553-6968a51537c7\") " pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:12.692191 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:12.692099    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/36732a46-1407-4b59-9553-6968a51537c7-online-tls\") pod \"feast-credit-scoring-9cc989665-755hx\" (UID: \"36732a46-1407-4b59-9553-6968a51537c7\") " pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:12.692191 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:12.692147    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/36732a46-1407-4b59-9553-6968a51537c7-registry-tls\") pod \"feast-credit-scoring-9cc989665-755hx\" (UID: \"36732a46-1407-4b59-9553-6968a51537c7\") " pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:12.692340 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:12.692266    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/36732a46-1407-4b59-9553-6968a51537c7-feast-data\") pod \"feast-credit-scoring-9cc989665-755hx\" (UID: \"36732a46-1407-4b59-9553-6968a51537c7\") " pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:12.692608 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:12.692583    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/36732a46-1407-4b59-9553-6968a51537c7-feast-data\") pod \"feast-credit-scoring-9cc989665-755hx\" (UID: \"36732a46-1407-4b59-9553-6968a51537c7\") " pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:12.694642 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:12.694618    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/36732a46-1407-4b59-9553-6968a51537c7-online-tls\") pod \"feast-credit-scoring-9cc989665-755hx\" (UID: \"36732a46-1407-4b59-9553-6968a51537c7\") " pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:12.694812 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:12.694792    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/36732a46-1407-4b59-9553-6968a51537c7-offline-tls\") pod \"feast-credit-scoring-9cc989665-755hx\" (UID: \"36732a46-1407-4b59-9553-6968a51537c7\") " pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:12.694853 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:12.694792    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/36732a46-1407-4b59-9553-6968a51537c7-registry-tls\") pod \"feast-credit-scoring-9cc989665-755hx\" (UID: \"36732a46-1407-4b59-9553-6968a51537c7\") " pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:12.699047 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:12.699028    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk6ql\" (UniqueName: \"kubernetes.io/projected/36732a46-1407-4b59-9553-6968a51537c7-kube-api-access-kk6ql\") pod \"feast-credit-scoring-9cc989665-755hx\" (UID: \"36732a46-1407-4b59-9553-6968a51537c7\") " pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:12.824401 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:12.824302    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:12.961064 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:12.961036    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-9cc989665-755hx"]
May 25 14:11:12.963619 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:11:12.963587    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36732a46_1407_4b59_9553_6968a51537c7.slice/crio-eb207e71c71cdc7617362a05b079d7884dc077716778b0606245d6b8829acfd1 WatchSource:0}: Error finding container eb207e71c71cdc7617362a05b079d7884dc077716778b0606245d6b8829acfd1: Status 404 returned error can't find the container with id eb207e71c71cdc7617362a05b079d7884dc077716778b0606245d6b8829acfd1
May 25 14:11:13.137464 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:13.137392    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx" event={"ID":"36732a46-1407-4b59-9553-6968a51537c7","Type":"ContainerStarted","Data":"704c5dbdadc6b1e1102d0eda8b3f89418954475dc8ba1e11516abc5aba815fa2"}
May 25 14:11:13.137464 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:13.137426    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx" event={"ID":"36732a46-1407-4b59-9553-6968a51537c7","Type":"ContainerStarted","Data":"eb207e71c71cdc7617362a05b079d7884dc077716778b0606245d6b8829acfd1"}
May 25 14:11:14.141130 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:14.141086    2571 generic.go:358] "Generic (PLEG): container finished" podID="36732a46-1407-4b59-9553-6968a51537c7" containerID="704c5dbdadc6b1e1102d0eda8b3f89418954475dc8ba1e11516abc5aba815fa2" exitCode=0
May 25 14:11:14.141608 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:14.141152    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx" event={"ID":"36732a46-1407-4b59-9553-6968a51537c7","Type":"ContainerDied","Data":"704c5dbdadc6b1e1102d0eda8b3f89418954475dc8ba1e11516abc5aba815fa2"}
May 25 14:11:15.146037 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:15.146007    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx" event={"ID":"36732a46-1407-4b59-9553-6968a51537c7","Type":"ContainerStarted","Data":"3cad5939ce5b024f4713a7c90f9ac92c61aa369e859ccc32de64ca376a0960cc"}
May 25 14:11:21.167217 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:21.167182    2571 generic.go:358] "Generic (PLEG): container finished" podID="36732a46-1407-4b59-9553-6968a51537c7" containerID="3cad5939ce5b024f4713a7c90f9ac92c61aa369e859ccc32de64ca376a0960cc" exitCode=0
May 25 14:11:21.167594 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:21.167233    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx" event={"ID":"36732a46-1407-4b59-9553-6968a51537c7","Type":"ContainerDied","Data":"3cad5939ce5b024f4713a7c90f9ac92c61aa369e859ccc32de64ca376a0960cc"}
May 25 14:11:22.174018 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:22.173960    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx" event={"ID":"36732a46-1407-4b59-9553-6968a51537c7","Type":"ContainerStarted","Data":"827ff903ac20154c8ba6fdfa01496e07146039ca9e8cfd0defe88269a7e864b8"}
May 25 14:11:22.174388 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:22.174034    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx" event={"ID":"36732a46-1407-4b59-9553-6968a51537c7","Type":"ContainerStarted","Data":"21d046a8dbb527691a2ffce6a589f680c38bc249c3c5d688697be7853a2facb6"}
May 25 14:11:22.174388 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:22.174050    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx" event={"ID":"36732a46-1407-4b59-9553-6968a51537c7","Type":"ContainerStarted","Data":"ba077ea687b9b2b820201c0a56569196774fbbdbefbcac61bd88fd1968b61b76"}
May 25 14:11:22.202383 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:22.202323    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx" podStartSLOduration=10.202305192 podStartE2EDuration="10.202305192s" podCreationTimestamp="2026-05-25 14:11:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 14:11:22.200436908 +0000 UTC m=+690.035293038" watchObservedRunningTime="2026-05-25 14:11:22.202305192 +0000 UTC m=+690.037161285"
May 25 14:11:22.827520 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:22.827475    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:22.827520 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:22.827530    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:22.827718 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:22.827544    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:24.827366 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:24.827327    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx" podUID="36732a46-1407-4b59-9553-6968a51537c7" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.28:8816: connect: connection refused"
May 25 14:11:24.827762 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:24.827360    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx" podUID="36732a46-1407-4b59-9553-6968a51537c7" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.28:6571: connect: connection refused"
May 25 14:11:24.827762 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:24.827476    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx" podUID="36732a46-1407-4b59-9553-6968a51537c7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused"
May 25 14:11:24.828557 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:24.828538    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:24.828614 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:24.828575    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:24.828614 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:24.828590    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:27.825356 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:27.825314    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx" podUID="36732a46-1407-4b59-9553-6968a51537c7" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.28:6571: connect: connection refused"
May 25 14:11:27.826246 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:27.826218    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:27.827259 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:27.827238    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:27.831011 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:27.830994    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:28.197068 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:28.197035    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:30.824698 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:30.824566    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx" podUID="36732a46-1407-4b59-9553-6968a51537c7" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.28:6571: connect: connection refused"
May 25 14:11:33.825180 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:33.825150    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:33.825903 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:33.825883    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:11:58.797878 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:58.797843    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-7947896bbc-j8l66"]
May 25 14:11:58.801033 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:58.801014    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:11:58.820002 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:58.819976    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7947896bbc-j8l66"]
May 25 14:11:58.905008 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:58.904979    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5-trusted-ca-bundle\") pod \"console-7947896bbc-j8l66\" (UID: \"df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5\") " pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:11:58.905126 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:58.905031    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5-console-oauth-config\") pod \"console-7947896bbc-j8l66\" (UID: \"df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5\") " pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:11:58.905126 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:58.905059    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5-oauth-serving-cert\") pod \"console-7947896bbc-j8l66\" (UID: \"df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5\") " pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:11:58.905126 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:58.905108    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5-service-ca\") pod \"console-7947896bbc-j8l66\" (UID: \"df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5\") " pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:11:58.905237 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:58.905157    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5-console-config\") pod \"console-7947896bbc-j8l66\" (UID: \"df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5\") " pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:11:58.905237 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:58.905176    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5-console-serving-cert\") pod \"console-7947896bbc-j8l66\" (UID: \"df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5\") " pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:11:58.905237 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:58.905201    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcz26\" (UniqueName: \"kubernetes.io/projected/df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5-kube-api-access-lcz26\") pod \"console-7947896bbc-j8l66\" (UID: \"df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5\") " pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:11:59.006240 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:59.006213    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5-service-ca\") pod \"console-7947896bbc-j8l66\" (UID: \"df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5\") " pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:11:59.006367 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:59.006255    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5-console-config\") pod \"console-7947896bbc-j8l66\" (UID: \"df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5\") " pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:11:59.006367 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:59.006276    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5-console-serving-cert\") pod \"console-7947896bbc-j8l66\" (UID: \"df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5\") " pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:11:59.006367 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:59.006293    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lcz26\" (UniqueName: \"kubernetes.io/projected/df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5-kube-api-access-lcz26\") pod \"console-7947896bbc-j8l66\" (UID: \"df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5\") " pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:11:59.006367 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:59.006324    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5-trusted-ca-bundle\") pod \"console-7947896bbc-j8l66\" (UID: \"df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5\") " pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:11:59.006367 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:59.006355    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5-console-oauth-config\") pod \"console-7947896bbc-j8l66\" (UID: \"df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5\") " pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:11:59.006585 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:59.006383    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5-oauth-serving-cert\") pod \"console-7947896bbc-j8l66\" (UID: \"df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5\") " pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:11:59.007031 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:59.006998    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5-console-config\") pod \"console-7947896bbc-j8l66\" (UID: \"df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5\") " pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:11:59.007144 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:59.007030    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5-service-ca\") pod \"console-7947896bbc-j8l66\" (UID: \"df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5\") " pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:11:59.007261 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:59.007183    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5-trusted-ca-bundle\") pod \"console-7947896bbc-j8l66\" (UID: \"df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5\") " pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:11:59.007261 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:59.007241    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5-oauth-serving-cert\") pod \"console-7947896bbc-j8l66\" (UID: \"df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5\") " pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:11:59.008896 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:59.008872    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5-console-serving-cert\") pod \"console-7947896bbc-j8l66\" (UID: \"df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5\") " pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:11:59.009005 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:59.008949    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5-console-oauth-config\") pod \"console-7947896bbc-j8l66\" (UID: \"df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5\") " pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:11:59.013875 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:59.013848    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcz26\" (UniqueName: \"kubernetes.io/projected/df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5-kube-api-access-lcz26\") pod \"console-7947896bbc-j8l66\" (UID: \"df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5\") " pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:11:59.110426 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:59.110355    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:11:59.321675 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:11:59.321648    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7947896bbc-j8l66"]
May 25 14:11:59.323848 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:11:59.323800    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf3be7c8_3d64_4712_a3ee_d9d7c4ba98b5.slice/crio-5ac03add3128b045cd0100bd19000ff5b68e8c6ea5f749934627e1f1a065d586 WatchSource:0}: Error finding container 5ac03add3128b045cd0100bd19000ff5b68e8c6ea5f749934627e1f1a065d586: Status 404 returned error can't find the container with id 5ac03add3128b045cd0100bd19000ff5b68e8c6ea5f749934627e1f1a065d586
May 25 14:12:00.297160 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:00.297110    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7947896bbc-j8l66" event={"ID":"df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5","Type":"ContainerStarted","Data":"9b4415b1d7efe9fab6959918adf9a9b8eaf320c02d95956a3ecb37ea1601b8dd"}
May 25 14:12:00.297531 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:00.297169    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7947896bbc-j8l66" event={"ID":"df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5","Type":"ContainerStarted","Data":"5ac03add3128b045cd0100bd19000ff5b68e8c6ea5f749934627e1f1a065d586"}
May 25 14:12:00.317445 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:00.317401    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-7947896bbc-j8l66" podStartSLOduration=2.317387206 podStartE2EDuration="2.317387206s" podCreationTimestamp="2026-05-25 14:11:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 14:12:00.315176101 +0000 UTC m=+728.150032194" watchObservedRunningTime="2026-05-25 14:12:00.317387206 +0000 UTC m=+728.152243299"
May 25 14:12:09.111005 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:09.110965    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:12:09.111005 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:09.111013    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:12:09.116675 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:09.116649    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:12:09.328785 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:09.328758    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7947896bbc-j8l66"
May 25 14:12:09.379849 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:09.379771    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57dcd5df8-lv4w2"]
May 25 14:12:33.459479 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:33.459442    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-9cc989665-755hx"]
May 25 14:12:33.460143 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:33.459900    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx" podUID="36732a46-1407-4b59-9553-6968a51537c7" containerName="registry" containerID="cri-o://ba077ea687b9b2b820201c0a56569196774fbbdbefbcac61bd88fd1968b61b76" gracePeriod=30
May 25 14:12:33.460143 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:33.460001    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx" podUID="36732a46-1407-4b59-9553-6968a51537c7" containerName="online" containerID="cri-o://21d046a8dbb527691a2ffce6a589f680c38bc249c3c5d688697be7853a2facb6" gracePeriod=30
May 25 14:12:33.460143 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:33.460025    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx" podUID="36732a46-1407-4b59-9553-6968a51537c7" containerName="offline" containerID="cri-o://827ff903ac20154c8ba6fdfa01496e07146039ca9e8cfd0defe88269a7e864b8" gracePeriod=30
May 25 14:12:33.484405 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:33.484377    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-pt8f6"]
May 25 14:12:33.484579 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:33.484547    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/postgres-7c7656cc6d-pt8f6" podUID="7a3f8f28-e07b-4bf7-a12a-fd034d8faba6" containerName="postgres" containerID="cri-o://65fc63beea848c38768f319ba883919a91f61b8c1da727d62f07e36d05919f20" gracePeriod=30
May 25 14:12:33.738572 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:33.738552    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-pt8f6"
May 25 14:12:33.809113 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:33.809072    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t482c\" (UniqueName: \"kubernetes.io/projected/7a3f8f28-e07b-4bf7-a12a-fd034d8faba6-kube-api-access-t482c\") pod \"7a3f8f28-e07b-4bf7-a12a-fd034d8faba6\" (UID: \"7a3f8f28-e07b-4bf7-a12a-fd034d8faba6\") "
May 25 14:12:33.809289 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:33.809172    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/7a3f8f28-e07b-4bf7-a12a-fd034d8faba6-postgresdata\") pod \"7a3f8f28-e07b-4bf7-a12a-fd034d8faba6\" (UID: \"7a3f8f28-e07b-4bf7-a12a-fd034d8faba6\") "
May 25 14:12:33.811281 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:33.811245    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a3f8f28-e07b-4bf7-a12a-fd034d8faba6-kube-api-access-t482c" (OuterVolumeSpecName: "kube-api-access-t482c") pod "7a3f8f28-e07b-4bf7-a12a-fd034d8faba6" (UID: "7a3f8f28-e07b-4bf7-a12a-fd034d8faba6"). InnerVolumeSpecName "kube-api-access-t482c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 14:12:33.838313 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:33.838279    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a3f8f28-e07b-4bf7-a12a-fd034d8faba6-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "7a3f8f28-e07b-4bf7-a12a-fd034d8faba6" (UID: "7a3f8f28-e07b-4bf7-a12a-fd034d8faba6"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 14:12:33.909787 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:33.909753    2571 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/7a3f8f28-e07b-4bf7-a12a-fd034d8faba6-postgresdata\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:12:33.909787 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:33.909783    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t482c\" (UniqueName: \"kubernetes.io/projected/7a3f8f28-e07b-4bf7-a12a-fd034d8faba6-kube-api-access-t482c\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:12:34.405299 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.405222    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-57dcd5df8-lv4w2" podUID="1707709d-e6b6-40f0-8167-6bbcd4bbb4ea" containerName="console" containerID="cri-o://6254e7acd60dbb585032bb60dd594fd4003032168d8c6b5105702041e4fdef2a" gracePeriod=15
May 25 14:12:34.406208 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.406181    2571 generic.go:358] "Generic (PLEG): container finished" podID="36732a46-1407-4b59-9553-6968a51537c7" containerID="21d046a8dbb527691a2ffce6a589f680c38bc249c3c5d688697be7853a2facb6" exitCode=0
May 25 14:12:34.406327 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.406261    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx" event={"ID":"36732a46-1407-4b59-9553-6968a51537c7","Type":"ContainerDied","Data":"21d046a8dbb527691a2ffce6a589f680c38bc249c3c5d688697be7853a2facb6"}
May 25 14:12:34.407391 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.407368    2571 generic.go:358] "Generic (PLEG): container finished" podID="7a3f8f28-e07b-4bf7-a12a-fd034d8faba6" containerID="65fc63beea848c38768f319ba883919a91f61b8c1da727d62f07e36d05919f20" exitCode=0
May 25 14:12:34.407491 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.407402    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-pt8f6" event={"ID":"7a3f8f28-e07b-4bf7-a12a-fd034d8faba6","Type":"ContainerDied","Data":"65fc63beea848c38768f319ba883919a91f61b8c1da727d62f07e36d05919f20"}
May 25 14:12:34.407491 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.407422    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-pt8f6"
May 25 14:12:34.407491 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.407426    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-pt8f6" event={"ID":"7a3f8f28-e07b-4bf7-a12a-fd034d8faba6","Type":"ContainerDied","Data":"9864b6fde48cf3bf2b577bb9dfb84cd79e39f9085238429340d6060500de0e0e"}
May 25 14:12:34.407491 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.407445    2571 scope.go:117] "RemoveContainer" containerID="65fc63beea848c38768f319ba883919a91f61b8c1da727d62f07e36d05919f20"
May 25 14:12:34.415761 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.415742    2571 scope.go:117] "RemoveContainer" containerID="65fc63beea848c38768f319ba883919a91f61b8c1da727d62f07e36d05919f20"
May 25 14:12:34.416049 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:12:34.416032    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65fc63beea848c38768f319ba883919a91f61b8c1da727d62f07e36d05919f20\": container with ID starting with 65fc63beea848c38768f319ba883919a91f61b8c1da727d62f07e36d05919f20 not found: ID does not exist" containerID="65fc63beea848c38768f319ba883919a91f61b8c1da727d62f07e36d05919f20"
May 25 14:12:34.416120 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.416056    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65fc63beea848c38768f319ba883919a91f61b8c1da727d62f07e36d05919f20"} err="failed to get container status \"65fc63beea848c38768f319ba883919a91f61b8c1da727d62f07e36d05919f20\": rpc error: code = NotFound desc = could not find container \"65fc63beea848c38768f319ba883919a91f61b8c1da727d62f07e36d05919f20\": container with ID starting with 65fc63beea848c38768f319ba883919a91f61b8c1da727d62f07e36d05919f20 not found: ID does not exist"
May 25 14:12:34.472877 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.472858    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-pt8f6"]
May 25 14:12:34.476103 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.476082    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-pt8f6"]
May 25 14:12:34.647978 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.647958    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-57dcd5df8-lv4w2_1707709d-e6b6-40f0-8167-6bbcd4bbb4ea/console/0.log"
May 25 14:12:34.648099 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.648027    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:12:34.717903 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.717879    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-console-serving-cert\") pod \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") "
May 25 14:12:34.718043 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.717984    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2x6t\" (UniqueName: \"kubernetes.io/projected/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-kube-api-access-d2x6t\") pod \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") "
May 25 14:12:34.718043 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.718019    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-trusted-ca-bundle\") pod \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") "
May 25 14:12:34.718153 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.718070    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-service-ca\") pod \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") "
May 25 14:12:34.718153 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.718097    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-oauth-serving-cert\") pod \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") "
May 25 14:12:34.718261 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.718140    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-console-oauth-config\") pod \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") "
May 25 14:12:34.718261 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.718210    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-console-config\") pod \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\" (UID: \"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea\") "
May 25 14:12:34.718550 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.718510    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-service-ca" (OuterVolumeSpecName: "service-ca") pod "1707709d-e6b6-40f0-8167-6bbcd4bbb4ea" (UID: "1707709d-e6b6-40f0-8167-6bbcd4bbb4ea"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:12:34.718550 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.718522    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1707709d-e6b6-40f0-8167-6bbcd4bbb4ea" (UID: "1707709d-e6b6-40f0-8167-6bbcd4bbb4ea"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:12:34.718550 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.718528    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1707709d-e6b6-40f0-8167-6bbcd4bbb4ea" (UID: "1707709d-e6b6-40f0-8167-6bbcd4bbb4ea"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:12:34.719019 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.718996    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-console-config" (OuterVolumeSpecName: "console-config") pod "1707709d-e6b6-40f0-8167-6bbcd4bbb4ea" (UID: "1707709d-e6b6-40f0-8167-6bbcd4bbb4ea"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:12:34.720316 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.720287    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1707709d-e6b6-40f0-8167-6bbcd4bbb4ea" (UID: "1707709d-e6b6-40f0-8167-6bbcd4bbb4ea"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:12:34.720405 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.720319    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-kube-api-access-d2x6t" (OuterVolumeSpecName: "kube-api-access-d2x6t") pod "1707709d-e6b6-40f0-8167-6bbcd4bbb4ea" (UID: "1707709d-e6b6-40f0-8167-6bbcd4bbb4ea"). InnerVolumeSpecName "kube-api-access-d2x6t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 14:12:34.720462 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.720441    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1707709d-e6b6-40f0-8167-6bbcd4bbb4ea" (UID: "1707709d-e6b6-40f0-8167-6bbcd4bbb4ea"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:12:34.818868 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.818842    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-console-serving-cert\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:12:34.818868 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.818866    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2x6t\" (UniqueName: \"kubernetes.io/projected/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-kube-api-access-d2x6t\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:12:34.819018 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.818877    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-trusted-ca-bundle\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:12:34.819018 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.818887    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-service-ca\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:12:34.819018 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.818895    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-oauth-serving-cert\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:12:34.819018 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.818903    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-console-oauth-config\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:12:34.819018 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.818912    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea-console-config\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:12:34.825685 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:34.825661    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a3f8f28-e07b-4bf7-a12a-fd034d8faba6" path="/var/lib/kubelet/pods/7a3f8f28-e07b-4bf7-a12a-fd034d8faba6/volumes"
May 25 14:12:35.412577 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:35.412551    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-57dcd5df8-lv4w2_1707709d-e6b6-40f0-8167-6bbcd4bbb4ea/console/0.log"
May 25 14:12:35.412772 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:35.412588    2571 generic.go:358] "Generic (PLEG): container finished" podID="1707709d-e6b6-40f0-8167-6bbcd4bbb4ea" containerID="6254e7acd60dbb585032bb60dd594fd4003032168d8c6b5105702041e4fdef2a" exitCode=2
May 25 14:12:35.412772 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:35.412620    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57dcd5df8-lv4w2" event={"ID":"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea","Type":"ContainerDied","Data":"6254e7acd60dbb585032bb60dd594fd4003032168d8c6b5105702041e4fdef2a"}
May 25 14:12:35.412772 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:35.412640    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57dcd5df8-lv4w2" event={"ID":"1707709d-e6b6-40f0-8167-6bbcd4bbb4ea","Type":"ContainerDied","Data":"936671e6a6f2ee51c6ca04330d2324043a5701e8197637a716fead5a09a6e92c"}
May 25 14:12:35.412772 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:35.412655    2571 scope.go:117] "RemoveContainer" containerID="6254e7acd60dbb585032bb60dd594fd4003032168d8c6b5105702041e4fdef2a"
May 25 14:12:35.412772 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:35.412662    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57dcd5df8-lv4w2"
May 25 14:12:35.420403 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:35.420386    2571 scope.go:117] "RemoveContainer" containerID="6254e7acd60dbb585032bb60dd594fd4003032168d8c6b5105702041e4fdef2a"
May 25 14:12:35.420670 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:12:35.420653    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6254e7acd60dbb585032bb60dd594fd4003032168d8c6b5105702041e4fdef2a\": container with ID starting with 6254e7acd60dbb585032bb60dd594fd4003032168d8c6b5105702041e4fdef2a not found: ID does not exist" containerID="6254e7acd60dbb585032bb60dd594fd4003032168d8c6b5105702041e4fdef2a"
May 25 14:12:35.420735 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:35.420677    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6254e7acd60dbb585032bb60dd594fd4003032168d8c6b5105702041e4fdef2a"} err="failed to get container status \"6254e7acd60dbb585032bb60dd594fd4003032168d8c6b5105702041e4fdef2a\": rpc error: code = NotFound desc = could not find container \"6254e7acd60dbb585032bb60dd594fd4003032168d8c6b5105702041e4fdef2a\": container with ID starting with 6254e7acd60dbb585032bb60dd594fd4003032168d8c6b5105702041e4fdef2a not found: ID does not exist"
May 25 14:12:35.433961 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:35.433917    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57dcd5df8-lv4w2"]
May 25 14:12:35.439604 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:35.439584    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57dcd5df8-lv4w2"]
May 25 14:12:36.826256 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:36.826221    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1707709d-e6b6-40f0-8167-6bbcd4bbb4ea" path="/var/lib/kubelet/pods/1707709d-e6b6-40f0-8167-6bbcd4bbb4ea/volumes"
May 25 14:12:38.193044 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:38.193006    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx" podUID="36732a46-1407-4b59-9553-6968a51537c7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused"
May 25 14:12:48.193782 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:48.193745    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx" podUID="36732a46-1407-4b59-9553-6968a51537c7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused"
May 25 14:12:58.193072 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:58.193027    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx" podUID="36732a46-1407-4b59-9553-6968a51537c7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused"
May 25 14:12:58.193546 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:12:58.193178    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:13:03.826074 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:03.825990    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx" podUID="36732a46-1407-4b59-9553-6968a51537c7" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.28:6571: connect: connection refused"
May 25 14:13:04.108482 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.108460    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:13:04.251838 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.251806    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/36732a46-1407-4b59-9553-6968a51537c7-online-tls\") pod \"36732a46-1407-4b59-9553-6968a51537c7\" (UID: \"36732a46-1407-4b59-9553-6968a51537c7\") "
May 25 14:13:04.251838 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.251842    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/36732a46-1407-4b59-9553-6968a51537c7-feast-data\") pod \"36732a46-1407-4b59-9553-6968a51537c7\" (UID: \"36732a46-1407-4b59-9553-6968a51537c7\") "
May 25 14:13:04.252192 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.251883    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/36732a46-1407-4b59-9553-6968a51537c7-registry-tls\") pod \"36732a46-1407-4b59-9553-6968a51537c7\" (UID: \"36732a46-1407-4b59-9553-6968a51537c7\") "
May 25 14:13:04.252192 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.251909    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/36732a46-1407-4b59-9553-6968a51537c7-offline-tls\") pod \"36732a46-1407-4b59-9553-6968a51537c7\" (UID: \"36732a46-1407-4b59-9553-6968a51537c7\") "
May 25 14:13:04.252192 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.252018    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kk6ql\" (UniqueName: \"kubernetes.io/projected/36732a46-1407-4b59-9553-6968a51537c7-kube-api-access-kk6ql\") pod \"36732a46-1407-4b59-9553-6968a51537c7\" (UID: \"36732a46-1407-4b59-9553-6968a51537c7\") "
May 25 14:13:04.254129 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.254088    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36732a46-1407-4b59-9553-6968a51537c7-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "36732a46-1407-4b59-9553-6968a51537c7" (UID: "36732a46-1407-4b59-9553-6968a51537c7"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:13:04.254251 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.254192    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36732a46-1407-4b59-9553-6968a51537c7-kube-api-access-kk6ql" (OuterVolumeSpecName: "kube-api-access-kk6ql") pod "36732a46-1407-4b59-9553-6968a51537c7" (UID: "36732a46-1407-4b59-9553-6968a51537c7"). InnerVolumeSpecName "kube-api-access-kk6ql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 14:13:04.254628 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.254602    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36732a46-1407-4b59-9553-6968a51537c7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "36732a46-1407-4b59-9553-6968a51537c7" (UID: "36732a46-1407-4b59-9553-6968a51537c7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:13:04.254722 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.254628    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36732a46-1407-4b59-9553-6968a51537c7-online-tls" (OuterVolumeSpecName: "online-tls") pod "36732a46-1407-4b59-9553-6968a51537c7" (UID: "36732a46-1407-4b59-9553-6968a51537c7"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:13:04.259744 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.259719    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36732a46-1407-4b59-9553-6968a51537c7-feast-data" (OuterVolumeSpecName: "feast-data") pod "36732a46-1407-4b59-9553-6968a51537c7" (UID: "36732a46-1407-4b59-9553-6968a51537c7"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 14:13:04.352634 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.352583    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kk6ql\" (UniqueName: \"kubernetes.io/projected/36732a46-1407-4b59-9553-6968a51537c7-kube-api-access-kk6ql\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:13:04.352634 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.352605    2571 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/36732a46-1407-4b59-9553-6968a51537c7-online-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:13:04.352634 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.352616    2571 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/36732a46-1407-4b59-9553-6968a51537c7-feast-data\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:13:04.352634 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.352624    2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/36732a46-1407-4b59-9553-6968a51537c7-registry-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:13:04.352634 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.352632    2571 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/36732a46-1407-4b59-9553-6968a51537c7-offline-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:13:04.503328 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.503288    2571 generic.go:358] "Generic (PLEG): container finished" podID="36732a46-1407-4b59-9553-6968a51537c7" containerID="827ff903ac20154c8ba6fdfa01496e07146039ca9e8cfd0defe88269a7e864b8" exitCode=137
May 25 14:13:04.503328 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.503318    2571 generic.go:358] "Generic (PLEG): container finished" podID="36732a46-1407-4b59-9553-6968a51537c7" containerID="ba077ea687b9b2b820201c0a56569196774fbbdbefbcac61bd88fd1968b61b76" exitCode=137
May 25 14:13:04.503471 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.503367    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx"
May 25 14:13:04.503471 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.503408    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx" event={"ID":"36732a46-1407-4b59-9553-6968a51537c7","Type":"ContainerDied","Data":"827ff903ac20154c8ba6fdfa01496e07146039ca9e8cfd0defe88269a7e864b8"}
May 25 14:13:04.503471 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.503444    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx" event={"ID":"36732a46-1407-4b59-9553-6968a51537c7","Type":"ContainerDied","Data":"ba077ea687b9b2b820201c0a56569196774fbbdbefbcac61bd88fd1968b61b76"}
May 25 14:13:04.503471 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.503461    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-9cc989665-755hx" event={"ID":"36732a46-1407-4b59-9553-6968a51537c7","Type":"ContainerDied","Data":"eb207e71c71cdc7617362a05b079d7884dc077716778b0606245d6b8829acfd1"}
May 25 14:13:04.503632 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.503476    2571 scope.go:117] "RemoveContainer" containerID="827ff903ac20154c8ba6fdfa01496e07146039ca9e8cfd0defe88269a7e864b8"
May 25 14:13:04.511556 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.511521    2571 scope.go:117] "RemoveContainer" containerID="21d046a8dbb527691a2ffce6a589f680c38bc249c3c5d688697be7853a2facb6"
May 25 14:13:04.518723 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.518707    2571 scope.go:117] "RemoveContainer" containerID="ba077ea687b9b2b820201c0a56569196774fbbdbefbcac61bd88fd1968b61b76"
May 25 14:13:04.525734 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.525713    2571 scope.go:117] "RemoveContainer" containerID="3cad5939ce5b024f4713a7c90f9ac92c61aa369e859ccc32de64ca376a0960cc"
May 25 14:13:04.532458 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.532441    2571 scope.go:117] "RemoveContainer" containerID="704c5dbdadc6b1e1102d0eda8b3f89418954475dc8ba1e11516abc5aba815fa2"
May 25 14:13:04.538905 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.538884    2571 scope.go:117] "RemoveContainer" containerID="827ff903ac20154c8ba6fdfa01496e07146039ca9e8cfd0defe88269a7e864b8"
May 25 14:13:04.539247 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:13:04.539230    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"827ff903ac20154c8ba6fdfa01496e07146039ca9e8cfd0defe88269a7e864b8\": container with ID starting with 827ff903ac20154c8ba6fdfa01496e07146039ca9e8cfd0defe88269a7e864b8 not found: ID does not exist" containerID="827ff903ac20154c8ba6fdfa01496e07146039ca9e8cfd0defe88269a7e864b8"
May 25 14:13:04.539323 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.539256    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"827ff903ac20154c8ba6fdfa01496e07146039ca9e8cfd0defe88269a7e864b8"} err="failed to get container status \"827ff903ac20154c8ba6fdfa01496e07146039ca9e8cfd0defe88269a7e864b8\": rpc error: code = NotFound desc = could not find container \"827ff903ac20154c8ba6fdfa01496e07146039ca9e8cfd0defe88269a7e864b8\": container with ID starting with 827ff903ac20154c8ba6fdfa01496e07146039ca9e8cfd0defe88269a7e864b8 not found: ID does not exist"
May 25 14:13:04.539323 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.539274    2571 scope.go:117] "RemoveContainer" containerID="21d046a8dbb527691a2ffce6a589f680c38bc249c3c5d688697be7853a2facb6"
May 25 14:13:04.539550 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:13:04.539532    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21d046a8dbb527691a2ffce6a589f680c38bc249c3c5d688697be7853a2facb6\": container with ID starting with 21d046a8dbb527691a2ffce6a589f680c38bc249c3c5d688697be7853a2facb6 not found: ID does not exist" containerID="21d046a8dbb527691a2ffce6a589f680c38bc249c3c5d688697be7853a2facb6"
May 25 14:13:04.539621 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.539560    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21d046a8dbb527691a2ffce6a589f680c38bc249c3c5d688697be7853a2facb6"} err="failed to get container status \"21d046a8dbb527691a2ffce6a589f680c38bc249c3c5d688697be7853a2facb6\": rpc error: code = NotFound desc = could not find container \"21d046a8dbb527691a2ffce6a589f680c38bc249c3c5d688697be7853a2facb6\": container with ID starting with 21d046a8dbb527691a2ffce6a589f680c38bc249c3c5d688697be7853a2facb6 not found: ID does not exist"
May 25 14:13:04.539621 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.539584    2571 scope.go:117] "RemoveContainer" containerID="ba077ea687b9b2b820201c0a56569196774fbbdbefbcac61bd88fd1968b61b76"
May 25 14:13:04.539828 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:13:04.539810    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba077ea687b9b2b820201c0a56569196774fbbdbefbcac61bd88fd1968b61b76\": container with ID starting with ba077ea687b9b2b820201c0a56569196774fbbdbefbcac61bd88fd1968b61b76 not found: ID does not exist" containerID="ba077ea687b9b2b820201c0a56569196774fbbdbefbcac61bd88fd1968b61b76"
May 25 14:13:04.539922 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.539829    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba077ea687b9b2b820201c0a56569196774fbbdbefbcac61bd88fd1968b61b76"} err="failed to get container status \"ba077ea687b9b2b820201c0a56569196774fbbdbefbcac61bd88fd1968b61b76\": rpc error: code = NotFound desc = could not find container \"ba077ea687b9b2b820201c0a56569196774fbbdbefbcac61bd88fd1968b61b76\": container with ID starting with ba077ea687b9b2b820201c0a56569196774fbbdbefbcac61bd88fd1968b61b76 not found: ID does not exist"
May 25 14:13:04.539922 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.539843    2571 scope.go:117] "RemoveContainer" containerID="3cad5939ce5b024f4713a7c90f9ac92c61aa369e859ccc32de64ca376a0960cc"
May 25 14:13:04.540687 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:13:04.540577    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cad5939ce5b024f4713a7c90f9ac92c61aa369e859ccc32de64ca376a0960cc\": container with ID starting with 3cad5939ce5b024f4713a7c90f9ac92c61aa369e859ccc32de64ca376a0960cc not found: ID does not exist" containerID="3cad5939ce5b024f4713a7c90f9ac92c61aa369e859ccc32de64ca376a0960cc"
May 25 14:13:04.540687 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.540607    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cad5939ce5b024f4713a7c90f9ac92c61aa369e859ccc32de64ca376a0960cc"} err="failed to get container status \"3cad5939ce5b024f4713a7c90f9ac92c61aa369e859ccc32de64ca376a0960cc\": rpc error: code = NotFound desc = could not find container \"3cad5939ce5b024f4713a7c90f9ac92c61aa369e859ccc32de64ca376a0960cc\": container with ID starting with 3cad5939ce5b024f4713a7c90f9ac92c61aa369e859ccc32de64ca376a0960cc not found: ID does not exist"
May 25 14:13:04.540687 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.540627    2571 scope.go:117] "RemoveContainer" containerID="704c5dbdadc6b1e1102d0eda8b3f89418954475dc8ba1e11516abc5aba815fa2"
May 25 14:13:04.540919 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:13:04.540901    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"704c5dbdadc6b1e1102d0eda8b3f89418954475dc8ba1e11516abc5aba815fa2\": container with ID starting with 704c5dbdadc6b1e1102d0eda8b3f89418954475dc8ba1e11516abc5aba815fa2 not found: ID does not exist" containerID="704c5dbdadc6b1e1102d0eda8b3f89418954475dc8ba1e11516abc5aba815fa2"
May 25 14:13:04.540994 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.540926    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"704c5dbdadc6b1e1102d0eda8b3f89418954475dc8ba1e11516abc5aba815fa2"} err="failed to get container status \"704c5dbdadc6b1e1102d0eda8b3f89418954475dc8ba1e11516abc5aba815fa2\": rpc error: code = NotFound desc = could not find container \"704c5dbdadc6b1e1102d0eda8b3f89418954475dc8ba1e11516abc5aba815fa2\": container with ID starting with 704c5dbdadc6b1e1102d0eda8b3f89418954475dc8ba1e11516abc5aba815fa2 not found: ID does not exist"
May 25 14:13:04.540994 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.540981    2571 scope.go:117] "RemoveContainer" containerID="827ff903ac20154c8ba6fdfa01496e07146039ca9e8cfd0defe88269a7e864b8"
May 25 14:13:04.541241 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.541225    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"827ff903ac20154c8ba6fdfa01496e07146039ca9e8cfd0defe88269a7e864b8"} err="failed to get container status \"827ff903ac20154c8ba6fdfa01496e07146039ca9e8cfd0defe88269a7e864b8\": rpc error: code = NotFound desc = could not find container \"827ff903ac20154c8ba6fdfa01496e07146039ca9e8cfd0defe88269a7e864b8\": container with ID starting with 827ff903ac20154c8ba6fdfa01496e07146039ca9e8cfd0defe88269a7e864b8 not found: ID does not exist"
May 25 14:13:04.541306 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.541241    2571 scope.go:117] "RemoveContainer" containerID="21d046a8dbb527691a2ffce6a589f680c38bc249c3c5d688697be7853a2facb6"
May 25 14:13:04.541437 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.541420    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21d046a8dbb527691a2ffce6a589f680c38bc249c3c5d688697be7853a2facb6"} err="failed to get container status \"21d046a8dbb527691a2ffce6a589f680c38bc249c3c5d688697be7853a2facb6\": rpc error: code = NotFound desc = could not find container \"21d046a8dbb527691a2ffce6a589f680c38bc249c3c5d688697be7853a2facb6\": container with ID starting with 21d046a8dbb527691a2ffce6a589f680c38bc249c3c5d688697be7853a2facb6 not found: ID does not exist"
May 25 14:13:04.541498 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.541438    2571 scope.go:117] "RemoveContainer" containerID="ba077ea687b9b2b820201c0a56569196774fbbdbefbcac61bd88fd1968b61b76"
May 25 14:13:04.541658 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.541635    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba077ea687b9b2b820201c0a56569196774fbbdbefbcac61bd88fd1968b61b76"} err="failed to get container status \"ba077ea687b9b2b820201c0a56569196774fbbdbefbcac61bd88fd1968b61b76\": rpc error: code = NotFound desc = could not find container \"ba077ea687b9b2b820201c0a56569196774fbbdbefbcac61bd88fd1968b61b76\": container with ID starting with ba077ea687b9b2b820201c0a56569196774fbbdbefbcac61bd88fd1968b61b76 not found: ID does not exist"
May 25 14:13:04.541717 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.541672    2571 scope.go:117] "RemoveContainer" containerID="3cad5939ce5b024f4713a7c90f9ac92c61aa369e859ccc32de64ca376a0960cc"
May 25 14:13:04.541923 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.541901    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cad5939ce5b024f4713a7c90f9ac92c61aa369e859ccc32de64ca376a0960cc"} err="failed to get container status \"3cad5939ce5b024f4713a7c90f9ac92c61aa369e859ccc32de64ca376a0960cc\": rpc error: code = NotFound desc = could not find container \"3cad5939ce5b024f4713a7c90f9ac92c61aa369e859ccc32de64ca376a0960cc\": container with ID starting with 3cad5939ce5b024f4713a7c90f9ac92c61aa369e859ccc32de64ca376a0960cc not found: ID does not exist"
May 25 14:13:04.542095 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.541923    2571 scope.go:117] "RemoveContainer" containerID="704c5dbdadc6b1e1102d0eda8b3f89418954475dc8ba1e11516abc5aba815fa2"
May 25 14:13:04.542273 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.542235    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"704c5dbdadc6b1e1102d0eda8b3f89418954475dc8ba1e11516abc5aba815fa2"} err="failed to get container status \"704c5dbdadc6b1e1102d0eda8b3f89418954475dc8ba1e11516abc5aba815fa2\": rpc error: code = NotFound desc = could not find container \"704c5dbdadc6b1e1102d0eda8b3f89418954475dc8ba1e11516abc5aba815fa2\": container with ID starting with 704c5dbdadc6b1e1102d0eda8b3f89418954475dc8ba1e11516abc5aba815fa2 not found: ID does not exist"
May 25 14:13:04.543480 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.543462    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-9cc989665-755hx"]
May 25 14:13:04.554887 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.554869    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-9cc989665-755hx"]
May 25 14:13:04.825383 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:13:04.825353    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36732a46-1407-4b59-9553-6968a51537c7" path="/var/lib/kubelet/pods/36732a46-1407-4b59-9553-6968a51537c7/volumes"
May 25 14:14:34.929473 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:34.929401    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-854544c579-b28rf"]
May 25 14:14:34.929920 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:34.929829    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a3f8f28-e07b-4bf7-a12a-fd034d8faba6" containerName="postgres"
May 25 14:14:34.929920 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:34.929846    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a3f8f28-e07b-4bf7-a12a-fd034d8faba6" containerName="postgres"
May 25 14:14:34.929920 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:34.929862    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36732a46-1407-4b59-9553-6968a51537c7" containerName="feast-apply"
May 25 14:14:34.929920 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:34.929870    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="36732a46-1407-4b59-9553-6968a51537c7" containerName="feast-apply"
May 25 14:14:34.929920 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:34.929886    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36732a46-1407-4b59-9553-6968a51537c7" containerName="feast-init"
May 25 14:14:34.929920 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:34.929895    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="36732a46-1407-4b59-9553-6968a51537c7" containerName="feast-init"
May 25 14:14:34.929920 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:34.929904    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36732a46-1407-4b59-9553-6968a51537c7" containerName="offline"
May 25 14:14:34.929920 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:34.929912    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="36732a46-1407-4b59-9553-6968a51537c7" containerName="offline"
May 25 14:14:34.930253 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:34.929923    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1707709d-e6b6-40f0-8167-6bbcd4bbb4ea" containerName="console"
May 25 14:14:34.930253 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:34.929957    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1707709d-e6b6-40f0-8167-6bbcd4bbb4ea" containerName="console"
May 25 14:14:34.930253 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:34.929975    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36732a46-1407-4b59-9553-6968a51537c7" containerName="online"
May 25 14:14:34.930253 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:34.929984    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="36732a46-1407-4b59-9553-6968a51537c7" containerName="online"
May 25 14:14:34.930253 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:34.930008    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36732a46-1407-4b59-9553-6968a51537c7" containerName="registry"
May 25 14:14:34.930253 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:34.930017    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="36732a46-1407-4b59-9553-6968a51537c7" containerName="registry"
May 25 14:14:34.930253 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:34.930089    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="1707709d-e6b6-40f0-8167-6bbcd4bbb4ea" containerName="console"
May 25 14:14:34.930253 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:34.930105    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="7a3f8f28-e07b-4bf7-a12a-fd034d8faba6" containerName="postgres"
May 25 14:14:34.930253 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:34.930115    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="36732a46-1407-4b59-9553-6968a51537c7" containerName="registry"
May 25 14:14:34.930253 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:34.930126    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="36732a46-1407-4b59-9553-6968a51537c7" containerName="online"
May 25 14:14:34.930253 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:34.930137    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="36732a46-1407-4b59-9553-6968a51537c7" containerName="offline"
May 25 14:14:34.932851 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:34.932831    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-b28rf"
May 25 14:14:34.935053 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:34.935032    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\""
May 25 14:14:34.935053 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:34.935054    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-lw7nd\""
May 25 14:14:34.935679 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:34.935659    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\""
May 25 14:14:34.949135 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:34.949114    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-854544c579-b28rf"]
May 25 14:14:35.026854 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:35.026825    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jww6f\" (UniqueName: \"kubernetes.io/projected/61bfe916-4e23-4d41-b201-d838dc778ad1-kube-api-access-jww6f\") pod \"redis-854544c579-b28rf\" (UID: \"61bfe916-4e23-4d41-b201-d838dc778ad1\") " pod="test-ns-feast-rest/redis-854544c579-b28rf"
May 25 14:14:35.127794 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:35.127763    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jww6f\" (UniqueName: \"kubernetes.io/projected/61bfe916-4e23-4d41-b201-d838dc778ad1-kube-api-access-jww6f\") pod \"redis-854544c579-b28rf\" (UID: \"61bfe916-4e23-4d41-b201-d838dc778ad1\") " pod="test-ns-feast-rest/redis-854544c579-b28rf"
May 25 14:14:35.135557 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:35.135527    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jww6f\" (UniqueName: \"kubernetes.io/projected/61bfe916-4e23-4d41-b201-d838dc778ad1-kube-api-access-jww6f\") pod \"redis-854544c579-b28rf\" (UID: \"61bfe916-4e23-4d41-b201-d838dc778ad1\") " pod="test-ns-feast-rest/redis-854544c579-b28rf"
May 25 14:14:35.271079 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:35.271055    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-b28rf"
May 25 14:14:35.396926 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:35.396862    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-854544c579-b28rf"]
May 25 14:14:35.399653 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:14:35.399628    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61bfe916_4e23_4d41_b201_d838dc778ad1.slice/crio-88db2d2d08feaf5f58ee586d66650c24d9ac694f3ce962f24aae8bc3ec6201cf WatchSource:0}: Error finding container 88db2d2d08feaf5f58ee586d66650c24d9ac694f3ce962f24aae8bc3ec6201cf: Status 404 returned error can't find the container with id 88db2d2d08feaf5f58ee586d66650c24d9ac694f3ce962f24aae8bc3ec6201cf
May 25 14:14:35.770050 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:35.770014    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-b28rf" event={"ID":"61bfe916-4e23-4d41-b201-d838dc778ad1","Type":"ContainerStarted","Data":"88db2d2d08feaf5f58ee586d66650c24d9ac694f3ce962f24aae8bc3ec6201cf"}
May 25 14:14:37.778577 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:37.778541    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-b28rf" event={"ID":"61bfe916-4e23-4d41-b201-d838dc778ad1","Type":"ContainerStarted","Data":"97908028da12dc2f16ec5c48506a0377b167edc4063682beb82c9708d56cfdbf"}
May 25 14:14:37.794402 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:37.794348    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/redis-854544c579-b28rf" podStartSLOduration=2.390222248 podStartE2EDuration="3.794334883s" podCreationTimestamp="2026-05-25 14:14:34 +0000 UTC" firstStartedPulling="2026-05-25 14:14:35.401330262 +0000 UTC m=+883.236186334" lastFinishedPulling="2026-05-25 14:14:36.805442898 +0000 UTC m=+884.640298969" observedRunningTime="2026-05-25 14:14:37.793218375 +0000 UTC m=+885.628074468" watchObservedRunningTime="2026-05-25 14:14:37.794334883 +0000 UTC m=+885.629190994"
May 25 14:14:40.384565 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:40.384532    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-rfpfg"]
May 25 14:14:40.387769 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:40.387753    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-rfpfg"
May 25 14:14:40.389792 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:40.389771    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\""
May 25 14:14:40.394794 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:40.394766    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-rfpfg"]
May 25 14:14:40.467895 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:40.467870    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/e5cc0f98-b50d-43e8-a15e-cebdee5e8470-postgresdata\") pod \"postgres-7c7656cc6d-rfpfg\" (UID: \"e5cc0f98-b50d-43e8-a15e-cebdee5e8470\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-rfpfg"
May 25 14:14:40.468034 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:40.467916    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txqxd\" (UniqueName: \"kubernetes.io/projected/e5cc0f98-b50d-43e8-a15e-cebdee5e8470-kube-api-access-txqxd\") pod \"postgres-7c7656cc6d-rfpfg\" (UID: \"e5cc0f98-b50d-43e8-a15e-cebdee5e8470\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-rfpfg"
May 25 14:14:40.568413 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:40.568387    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-txqxd\" (UniqueName: \"kubernetes.io/projected/e5cc0f98-b50d-43e8-a15e-cebdee5e8470-kube-api-access-txqxd\") pod \"postgres-7c7656cc6d-rfpfg\" (UID: \"e5cc0f98-b50d-43e8-a15e-cebdee5e8470\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-rfpfg"
May 25 14:14:40.568534 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:40.568480    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/e5cc0f98-b50d-43e8-a15e-cebdee5e8470-postgresdata\") pod \"postgres-7c7656cc6d-rfpfg\" (UID: \"e5cc0f98-b50d-43e8-a15e-cebdee5e8470\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-rfpfg"
May 25 14:14:40.568824 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:40.568805    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/e5cc0f98-b50d-43e8-a15e-cebdee5e8470-postgresdata\") pod \"postgres-7c7656cc6d-rfpfg\" (UID: \"e5cc0f98-b50d-43e8-a15e-cebdee5e8470\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-rfpfg"
May 25 14:14:40.576730 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:40.576703    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txqxd\" (UniqueName: \"kubernetes.io/projected/e5cc0f98-b50d-43e8-a15e-cebdee5e8470-kube-api-access-txqxd\") pod \"postgres-7c7656cc6d-rfpfg\" (UID: \"e5cc0f98-b50d-43e8-a15e-cebdee5e8470\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-rfpfg"
May 25 14:14:40.698474 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:40.698453    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-rfpfg"
May 25 14:14:40.825890 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:14:40.825857    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode5cc0f98_b50d_43e8_a15e_cebdee5e8470.slice/crio-14485897cf43694430599a7a0ba600853aea0cb5e2d6a06f606a1a82cd49df17 WatchSource:0}: Error finding container 14485897cf43694430599a7a0ba600853aea0cb5e2d6a06f606a1a82cd49df17: Status 404 returned error can't find the container with id 14485897cf43694430599a7a0ba600853aea0cb5e2d6a06f606a1a82cd49df17
May 25 14:14:40.826290 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:40.826266    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-rfpfg"]
May 25 14:14:41.792474 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:41.792438    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-rfpfg" event={"ID":"e5cc0f98-b50d-43e8-a15e-cebdee5e8470","Type":"ContainerStarted","Data":"dad6bec88295526c8f300e33fb68ff33892f6877145067fd7cac096d03ca4103"}
May 25 14:14:41.792474 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:41.792474    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-rfpfg" event={"ID":"e5cc0f98-b50d-43e8-a15e-cebdee5e8470","Type":"ContainerStarted","Data":"14485897cf43694430599a7a0ba600853aea0cb5e2d6a06f606a1a82cd49df17"}
May 25 14:14:41.810556 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:41.810507    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-7c7656cc6d-rfpfg" podStartSLOduration=1.810492816 podStartE2EDuration="1.810492816s" podCreationTimestamp="2026-05-25 14:14:40 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 14:14:41.80835596 +0000 UTC m=+889.643212053" watchObservedRunningTime="2026-05-25 14:14:41.810492816 +0000 UTC m=+889.645348909"
May 25 14:14:46.073062 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:46.073028    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"]
May 25 14:14:46.076593 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:46.076574    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"
May 25 14:14:46.078413 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:46.078390    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\""
May 25 14:14:46.078523 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:46.078487    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\""
May 25 14:14:46.078617 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:46.078601    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\""
May 25 14:14:46.078806 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:46.078783    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-hpjhc\""
May 25 14:14:46.085833 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:46.085810    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"]
May 25 14:14:46.215138 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:46.215114    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8nq2\" (UniqueName: \"kubernetes.io/projected/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-kube-api-access-s8nq2\") pod \"feast-credit-scoring-5c95685c9-r2j9p\" (UID: \"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372\") " pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"
May 25 14:14:46.215238 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:46.215149    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-registry-tls\") pod \"feast-credit-scoring-5c95685c9-r2j9p\" (UID: \"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372\") " pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"
May 25 14:14:46.215238 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:46.215173    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-feast-data\") pod \"feast-credit-scoring-5c95685c9-r2j9p\" (UID: \"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372\") " pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"
May 25 14:14:46.215313 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:46.215255    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-online-tls\") pod \"feast-credit-scoring-5c95685c9-r2j9p\" (UID: \"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372\") " pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"
May 25 14:14:46.316638 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:46.316615    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-online-tls\") pod \"feast-credit-scoring-5c95685c9-r2j9p\" (UID: \"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372\") " pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"
May 25 14:14:46.316729 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:46.316679    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s8nq2\" (UniqueName: \"kubernetes.io/projected/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-kube-api-access-s8nq2\") pod \"feast-credit-scoring-5c95685c9-r2j9p\" (UID: \"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372\") " pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"
May 25 14:14:46.316729 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:46.316703    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-registry-tls\") pod \"feast-credit-scoring-5c95685c9-r2j9p\" (UID: \"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372\") " pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"
May 25 14:14:46.316729 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:46.316721    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-feast-data\") pod \"feast-credit-scoring-5c95685c9-r2j9p\" (UID: \"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372\") " pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"
May 25 14:14:46.316884 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:14:46.316746    2571 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-credit-scoring-online-tls: secret "feast-credit-scoring-online-tls" not found
May 25 14:14:46.316884 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:14:46.316816    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-online-tls podName:ba4fcc3a-c3f6-478d-b2d3-079be6b3d372 nodeName:}" failed. No retries permitted until 2026-05-25 14:14:46.816798237 +0000 UTC m=+894.651654310 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-online-tls") pod "feast-credit-scoring-5c95685c9-r2j9p" (UID: "ba4fcc3a-c3f6-478d-b2d3-079be6b3d372") : secret "feast-credit-scoring-online-tls" not found
May 25 14:14:46.317070 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:46.317054    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-feast-data\") pod \"feast-credit-scoring-5c95685c9-r2j9p\" (UID: \"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372\") " pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"
May 25 14:14:46.319249 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:46.319228    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-registry-tls\") pod \"feast-credit-scoring-5c95685c9-r2j9p\" (UID: \"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372\") " pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"
May 25 14:14:46.325028 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:46.324967    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8nq2\" (UniqueName: \"kubernetes.io/projected/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-kube-api-access-s8nq2\") pod \"feast-credit-scoring-5c95685c9-r2j9p\" (UID: \"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372\") " pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"
May 25 14:14:46.820811 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:46.820780    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-online-tls\") pod \"feast-credit-scoring-5c95685c9-r2j9p\" (UID: \"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372\") " pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"
May 25 14:14:46.823267 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:46.823243    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-online-tls\") pod \"feast-credit-scoring-5c95685c9-r2j9p\" (UID: \"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372\") " pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"
May 25 14:14:46.986356 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:46.986325    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"
May 25 14:14:47.125377 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:47.125353    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"]
May 25 14:14:47.126503 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:14:47.126475    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba4fcc3a_c3f6_478d_b2d3_079be6b3d372.slice/crio-956ae57cc547c2a79a0e372295ccbb3eae818ebe7fe4b33232de526fb973f261 WatchSource:0}: Error finding container 956ae57cc547c2a79a0e372295ccbb3eae818ebe7fe4b33232de526fb973f261: Status 404 returned error can't find the container with id 956ae57cc547c2a79a0e372295ccbb3eae818ebe7fe4b33232de526fb973f261
May 25 14:14:47.811899 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:47.811861    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p" event={"ID":"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372","Type":"ContainerStarted","Data":"2e973f678aad835b173e1fcbfc20890ebe2595cd25d1a35b69650367f09021dd"}
May 25 14:14:47.812135 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:47.811906    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p" event={"ID":"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372","Type":"ContainerStarted","Data":"956ae57cc547c2a79a0e372295ccbb3eae818ebe7fe4b33232de526fb973f261"}
May 25 14:14:48.816844 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:48.816807    2571 generic.go:358] "Generic (PLEG): container finished" podID="ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" containerID="2e973f678aad835b173e1fcbfc20890ebe2595cd25d1a35b69650367f09021dd" exitCode=0
May 25 14:14:48.817280 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:48.816889    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p" event={"ID":"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372","Type":"ContainerDied","Data":"2e973f678aad835b173e1fcbfc20890ebe2595cd25d1a35b69650367f09021dd"}
May 25 14:14:49.821714 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:49.821674    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p" event={"ID":"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372","Type":"ContainerStarted","Data":"278f778c4b6cb97db42653e78a4e2694e5376a44aa24eeabebb10bb6187c4609"}
May 25 14:14:55.841658 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:55.841626    2571 generic.go:358] "Generic (PLEG): container finished" podID="ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" containerID="278f778c4b6cb97db42653e78a4e2694e5376a44aa24eeabebb10bb6187c4609" exitCode=0
May 25 14:14:55.841989 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:55.841703    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p" event={"ID":"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372","Type":"ContainerDied","Data":"278f778c4b6cb97db42653e78a4e2694e5376a44aa24eeabebb10bb6187c4609"}
May 25 14:14:55.844669 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:55.844651    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\""
May 25 14:14:56.846689 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:56.846650    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p" event={"ID":"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372","Type":"ContainerStarted","Data":"b287807a323c6835436cfcc26889f29ee11c25f1ed97aab1dee15802cfd4d5f2"}
May 25 14:14:56.847072 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:56.846701    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p" event={"ID":"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372","Type":"ContainerStarted","Data":"cfa319d25dad5e4c3ebd542ee3c059e7340c71bf2dd82577dd5d8c09ada3f0bd"}
May 25 14:14:56.869729 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:56.869686    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p" podStartSLOduration=10.869673326000001 podStartE2EDuration="10.869673326s" podCreationTimestamp="2026-05-25 14:14:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 14:14:56.868319109 +0000 UTC m=+904.703175194" watchObservedRunningTime="2026-05-25 14:14:56.869673326 +0000 UTC m=+904.704529419"
May 25 14:14:56.986776 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:56.986737    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"
May 25 14:14:56.986776 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:56.986783    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"
May 25 14:14:58.986786 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:58.986745    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"
May 25 14:14:58.986786 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:58.986792    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"
May 25 14:14:58.988490 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:58.988457    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p" podUID="ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused"
May 25 14:14:58.988490 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:14:58.988479    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p" podUID="ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
May 25 14:15:01.986709 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:01.986657    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p" podUID="ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused"
May 25 14:15:01.992000 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:01.991972    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"
May 25 14:15:01.995113 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:01.995090    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"
May 25 14:15:04.987012 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:04.986962    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p" podUID="ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused"
May 25 14:15:07.987757 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:07.987725    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"
May 25 14:15:07.988450 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:07.988431    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"
May 25 14:15:12.000513 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:12.000484    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"]
May 25 14:15:12.004727 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:12.004708    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"
May 25 14:15:12.006909 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:12.006882    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\""
May 25 14:15:12.007048 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:12.006886    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\""
May 25 14:15:12.007048 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:12.006947    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-j4zds\""
May 25 14:15:12.007048 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:12.006886    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\""
May 25 14:15:12.015066 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:12.015045    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"]
May 25 14:15:12.026972 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:12.026946    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7dd66b89-f173-4ed2-a2c4-9de8466177d7-online-tls\") pod \"feast-driver-ranking-7784d54cb8-g75jg\" (UID: \"7dd66b89-f173-4ed2-a2c4-9de8466177d7\") " pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"
May 25 14:15:12.027113 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:12.027095    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7dd66b89-f173-4ed2-a2c4-9de8466177d7-feast-data\") pod \"feast-driver-ranking-7784d54cb8-g75jg\" (UID: \"7dd66b89-f173-4ed2-a2c4-9de8466177d7\") " pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"
May 25 14:15:12.027191 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:12.027147    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7dd66b89-f173-4ed2-a2c4-9de8466177d7-registry-tls\") pod \"feast-driver-ranking-7784d54cb8-g75jg\" (UID: \"7dd66b89-f173-4ed2-a2c4-9de8466177d7\") " pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"
May 25 14:15:12.027241 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:12.027195    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtxk5\" (UniqueName: \"kubernetes.io/projected/7dd66b89-f173-4ed2-a2c4-9de8466177d7-kube-api-access-jtxk5\") pod \"feast-driver-ranking-7784d54cb8-g75jg\" (UID: \"7dd66b89-f173-4ed2-a2c4-9de8466177d7\") " pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"
May 25 14:15:12.127609 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:12.127588    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7dd66b89-f173-4ed2-a2c4-9de8466177d7-online-tls\") pod \"feast-driver-ranking-7784d54cb8-g75jg\" (UID: \"7dd66b89-f173-4ed2-a2c4-9de8466177d7\") " pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"
May 25 14:15:12.127706 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:12.127640    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7dd66b89-f173-4ed2-a2c4-9de8466177d7-feast-data\") pod \"feast-driver-ranking-7784d54cb8-g75jg\" (UID: \"7dd66b89-f173-4ed2-a2c4-9de8466177d7\") " pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"
May 25 14:15:12.127706 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:12.127669    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7dd66b89-f173-4ed2-a2c4-9de8466177d7-registry-tls\") pod \"feast-driver-ranking-7784d54cb8-g75jg\" (UID: \"7dd66b89-f173-4ed2-a2c4-9de8466177d7\") " pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"
May 25 14:15:12.127706 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:12.127686    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jtxk5\" (UniqueName: \"kubernetes.io/projected/7dd66b89-f173-4ed2-a2c4-9de8466177d7-kube-api-access-jtxk5\") pod \"feast-driver-ranking-7784d54cb8-g75jg\" (UID: \"7dd66b89-f173-4ed2-a2c4-9de8466177d7\") " pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"
May 25 14:15:12.128129 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:12.128098    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7dd66b89-f173-4ed2-a2c4-9de8466177d7-feast-data\") pod \"feast-driver-ranking-7784d54cb8-g75jg\" (UID: \"7dd66b89-f173-4ed2-a2c4-9de8466177d7\") " pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"
May 25 14:15:12.130171 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:12.130150    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7dd66b89-f173-4ed2-a2c4-9de8466177d7-registry-tls\") pod \"feast-driver-ranking-7784d54cb8-g75jg\" (UID: \"7dd66b89-f173-4ed2-a2c4-9de8466177d7\") " pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"
May 25 14:15:12.130254 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:12.130190    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7dd66b89-f173-4ed2-a2c4-9de8466177d7-online-tls\") pod \"feast-driver-ranking-7784d54cb8-g75jg\" (UID: \"7dd66b89-f173-4ed2-a2c4-9de8466177d7\") " pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"
May 25 14:15:12.135769 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:12.135747    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtxk5\" (UniqueName: \"kubernetes.io/projected/7dd66b89-f173-4ed2-a2c4-9de8466177d7-kube-api-access-jtxk5\") pod \"feast-driver-ranking-7784d54cb8-g75jg\" (UID: \"7dd66b89-f173-4ed2-a2c4-9de8466177d7\") " pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"
May 25 14:15:12.313843 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:12.313763    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"
May 25 14:15:12.444118 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:12.444096    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"]
May 25 14:15:12.446211 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:15:12.446184    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dd66b89_f173_4ed2_a2c4_9de8466177d7.slice/crio-ca70c354e761900b91677834f0102d573640a11e9c2ab65b2b9f3d72cc640ddc WatchSource:0}: Error finding container ca70c354e761900b91677834f0102d573640a11e9c2ab65b2b9f3d72cc640ddc: Status 404 returned error can't find the container with id ca70c354e761900b91677834f0102d573640a11e9c2ab65b2b9f3d72cc640ddc
May 25 14:15:12.897125 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:12.897093    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg" event={"ID":"7dd66b89-f173-4ed2-a2c4-9de8466177d7","Type":"ContainerStarted","Data":"bf8888d593bd68fc4633b2d5d14f95ea8de0bba943b42f258bf0d44d6fa58fda"}
May 25 14:15:12.897125 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:12.897124    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg" event={"ID":"7dd66b89-f173-4ed2-a2c4-9de8466177d7","Type":"ContainerStarted","Data":"ca70c354e761900b91677834f0102d573640a11e9c2ab65b2b9f3d72cc640ddc"}
May 25 14:15:16.910874 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:16.910845    2571 generic.go:358] "Generic (PLEG): container finished" podID="7dd66b89-f173-4ed2-a2c4-9de8466177d7" containerID="bf8888d593bd68fc4633b2d5d14f95ea8de0bba943b42f258bf0d44d6fa58fda" exitCode=0
May 25 14:15:16.911167 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:16.910913    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg" event={"ID":"7dd66b89-f173-4ed2-a2c4-9de8466177d7","Type":"ContainerDied","Data":"bf8888d593bd68fc4633b2d5d14f95ea8de0bba943b42f258bf0d44d6fa58fda"}
May 25 14:15:17.915563 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:17.915526    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg" event={"ID":"7dd66b89-f173-4ed2-a2c4-9de8466177d7","Type":"ContainerStarted","Data":"8a6242342de44bafee987ce80f4f00e72304eb1548a798131145a376a200cb67"}
May 25 14:15:23.934222 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:23.934190    2571 generic.go:358] "Generic (PLEG): container finished" podID="7dd66b89-f173-4ed2-a2c4-9de8466177d7" containerID="8a6242342de44bafee987ce80f4f00e72304eb1548a798131145a376a200cb67" exitCode=0
May 25 14:15:23.934538 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:23.934259    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg" event={"ID":"7dd66b89-f173-4ed2-a2c4-9de8466177d7","Type":"ContainerDied","Data":"8a6242342de44bafee987ce80f4f00e72304eb1548a798131145a376a200cb67"}
May 25 14:15:24.940257 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:24.940222    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg" event={"ID":"7dd66b89-f173-4ed2-a2c4-9de8466177d7","Type":"ContainerStarted","Data":"62763710ecb049e597a7ead5537d48667dc16eb82cc042615d87b250acdcf7af"}
May 25 14:15:24.940257 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:24.940258    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg" event={"ID":"7dd66b89-f173-4ed2-a2c4-9de8466177d7","Type":"ContainerStarted","Data":"c815bb8516f4976915b8f526ca7ed804f2e74fe5c69f881e95e44ebb4ae31996"}
May 25 14:15:24.964114 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:24.964057    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg" podStartSLOduration=13.964039369 podStartE2EDuration="13.964039369s" podCreationTimestamp="2026-05-25 14:15:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 14:15:24.962210967 +0000 UTC m=+932.797067073" watchObservedRunningTime="2026-05-25 14:15:24.964039369 +0000 UTC m=+932.798895462"
May 25 14:15:27.314703 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:27.314662    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"
May 25 14:15:27.315182 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:27.314715    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"
May 25 14:15:27.316018 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:27.315986    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg" podUID="7dd66b89-f173-4ed2-a2c4-9de8466177d7" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.33:6571: connect: connection refused"
May 25 14:15:27.316119 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:27.316012    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg" podUID="7dd66b89-f173-4ed2-a2c4-9de8466177d7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused"
May 25 14:15:30.314526 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:30.314478    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg" podUID="7dd66b89-f173-4ed2-a2c4-9de8466177d7" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.33:6571: connect: connection refused"
May 25 14:15:30.319658 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:30.319633    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"
May 25 14:15:30.320079 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:30.320050    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"
May 25 14:15:30.322770 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:30.322743    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"
May 25 14:15:32.314099 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:32.314070    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"
May 25 14:15:33.314384 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:33.314337    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg" podUID="7dd66b89-f173-4ed2-a2c4-9de8466177d7" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.33:6571: connect: connection refused"
May 25 14:15:36.315550 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:36.315516    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"
May 25 14:15:36.317986 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:15:36.316258    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"
May 25 14:16:51.085127 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:51.085094    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"]
May 25 14:16:51.085640 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:51.085469    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p" podUID="ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" containerName="registry" containerID="cri-o://cfa319d25dad5e4c3ebd542ee3c059e7340c71bf2dd82577dd5d8c09ada3f0bd" gracePeriod=30
May 25 14:16:51.085640 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:51.085492    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p" podUID="ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" containerName="online" containerID="cri-o://b287807a323c6835436cfcc26889f29ee11c25f1ed97aab1dee15802cfd4d5f2" gracePeriod=30
May 25 14:16:51.091694 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:51.091668    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"]
May 25 14:16:51.092045 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:51.092005    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg" podUID="7dd66b89-f173-4ed2-a2c4-9de8466177d7" containerName="registry" containerID="cri-o://c815bb8516f4976915b8f526ca7ed804f2e74fe5c69f881e95e44ebb4ae31996" gracePeriod=30
May 25 14:16:51.092157 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:51.092006    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg" podUID="7dd66b89-f173-4ed2-a2c4-9de8466177d7" containerName="online" containerID="cri-o://62763710ecb049e597a7ead5537d48667dc16eb82cc042615d87b250acdcf7af" gracePeriod=30
May 25 14:16:51.992676 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:51.992599    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p" podUID="ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
May 25 14:16:52.233538 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:52.233504    2571 generic.go:358] "Generic (PLEG): container finished" podID="ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" containerID="b287807a323c6835436cfcc26889f29ee11c25f1ed97aab1dee15802cfd4d5f2" exitCode=0
May 25 14:16:52.233908 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:52.233568    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p" event={"ID":"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372","Type":"ContainerDied","Data":"b287807a323c6835436cfcc26889f29ee11c25f1ed97aab1dee15802cfd4d5f2"}
May 25 14:16:52.235595 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:52.235571    2571 generic.go:358] "Generic (PLEG): container finished" podID="7dd66b89-f173-4ed2-a2c4-9de8466177d7" containerID="62763710ecb049e597a7ead5537d48667dc16eb82cc042615d87b250acdcf7af" exitCode=0
May 25 14:16:52.235693 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:52.235615    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg" event={"ID":"7dd66b89-f173-4ed2-a2c4-9de8466177d7","Type":"ContainerDied","Data":"62763710ecb049e597a7ead5537d48667dc16eb82cc042615d87b250acdcf7af"}
May 25 14:16:52.746487 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:52.746453    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-rfpfg"]
May 25 14:16:52.746670 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:52.746640    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-7c7656cc6d-rfpfg" podUID="e5cc0f98-b50d-43e8-a15e-cebdee5e8470" containerName="postgres" containerID="cri-o://dad6bec88295526c8f300e33fb68ff33892f6877145067fd7cac096d03ca4103" gracePeriod=30
May 25 14:16:52.751126 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:52.751100    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-854544c579-b28rf"]
May 25 14:16:52.751305 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:52.751271    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/redis-854544c579-b28rf" podUID="61bfe916-4e23-4d41-b201-d838dc778ad1" containerName="redis" containerID="cri-o://97908028da12dc2f16ec5c48506a0377b167edc4063682beb82c9708d56cfdbf" gracePeriod=30
May 25 14:16:52.988147 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:52.988123    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-rfpfg"
May 25 14:16:53.017680 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.017655    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-b28rf"
May 25 14:16:53.174334 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.174308    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txqxd\" (UniqueName: \"kubernetes.io/projected/e5cc0f98-b50d-43e8-a15e-cebdee5e8470-kube-api-access-txqxd\") pod \"e5cc0f98-b50d-43e8-a15e-cebdee5e8470\" (UID: \"e5cc0f98-b50d-43e8-a15e-cebdee5e8470\") "
May 25 14:16:53.174500 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.174376    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/e5cc0f98-b50d-43e8-a15e-cebdee5e8470-postgresdata\") pod \"e5cc0f98-b50d-43e8-a15e-cebdee5e8470\" (UID: \"e5cc0f98-b50d-43e8-a15e-cebdee5e8470\") "
May 25 14:16:53.174565 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.174538    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jww6f\" (UniqueName: \"kubernetes.io/projected/61bfe916-4e23-4d41-b201-d838dc778ad1-kube-api-access-jww6f\") pod \"61bfe916-4e23-4d41-b201-d838dc778ad1\" (UID: \"61bfe916-4e23-4d41-b201-d838dc778ad1\") "
May 25 14:16:53.176607 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.176579    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5cc0f98-b50d-43e8-a15e-cebdee5e8470-kube-api-access-txqxd" (OuterVolumeSpecName: "kube-api-access-txqxd") pod "e5cc0f98-b50d-43e8-a15e-cebdee5e8470" (UID: "e5cc0f98-b50d-43e8-a15e-cebdee5e8470"). InnerVolumeSpecName "kube-api-access-txqxd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 14:16:53.176729 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.176652    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61bfe916-4e23-4d41-b201-d838dc778ad1-kube-api-access-jww6f" (OuterVolumeSpecName: "kube-api-access-jww6f") pod "61bfe916-4e23-4d41-b201-d838dc778ad1" (UID: "61bfe916-4e23-4d41-b201-d838dc778ad1"). InnerVolumeSpecName "kube-api-access-jww6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 14:16:53.203503 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.203474    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5cc0f98-b50d-43e8-a15e-cebdee5e8470-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "e5cc0f98-b50d-43e8-a15e-cebdee5e8470" (UID: "e5cc0f98-b50d-43e8-a15e-cebdee5e8470"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 14:16:53.239973 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.239924    2571 generic.go:358] "Generic (PLEG): container finished" podID="61bfe916-4e23-4d41-b201-d838dc778ad1" containerID="97908028da12dc2f16ec5c48506a0377b167edc4063682beb82c9708d56cfdbf" exitCode=0
May 25 14:16:53.240332 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.239991    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-b28rf"
May 25 14:16:53.240332 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.240039    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-b28rf" event={"ID":"61bfe916-4e23-4d41-b201-d838dc778ad1","Type":"ContainerDied","Data":"97908028da12dc2f16ec5c48506a0377b167edc4063682beb82c9708d56cfdbf"}
May 25 14:16:53.240332 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.240072    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-b28rf" event={"ID":"61bfe916-4e23-4d41-b201-d838dc778ad1","Type":"ContainerDied","Data":"88db2d2d08feaf5f58ee586d66650c24d9ac694f3ce962f24aae8bc3ec6201cf"}
May 25 14:16:53.240332 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.240086    2571 scope.go:117] "RemoveContainer" containerID="97908028da12dc2f16ec5c48506a0377b167edc4063682beb82c9708d56cfdbf"
May 25 14:16:53.241136 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.241116    2571 generic.go:358] "Generic (PLEG): container finished" podID="e5cc0f98-b50d-43e8-a15e-cebdee5e8470" containerID="dad6bec88295526c8f300e33fb68ff33892f6877145067fd7cac096d03ca4103" exitCode=0
May 25 14:16:53.241193 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.241157    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-rfpfg" event={"ID":"e5cc0f98-b50d-43e8-a15e-cebdee5e8470","Type":"ContainerDied","Data":"dad6bec88295526c8f300e33fb68ff33892f6877145067fd7cac096d03ca4103"}
May 25 14:16:53.241193 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.241175    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-rfpfg" event={"ID":"e5cc0f98-b50d-43e8-a15e-cebdee5e8470","Type":"ContainerDied","Data":"14485897cf43694430599a7a0ba600853aea0cb5e2d6a06f606a1a82cd49df17"}
May 25 14:16:53.241193 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.241175    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-rfpfg"
May 25 14:16:53.248690 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.248673    2571 scope.go:117] "RemoveContainer" containerID="97908028da12dc2f16ec5c48506a0377b167edc4063682beb82c9708d56cfdbf"
May 25 14:16:53.248965 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:16:53.248926    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97908028da12dc2f16ec5c48506a0377b167edc4063682beb82c9708d56cfdbf\": container with ID starting with 97908028da12dc2f16ec5c48506a0377b167edc4063682beb82c9708d56cfdbf not found: ID does not exist" containerID="97908028da12dc2f16ec5c48506a0377b167edc4063682beb82c9708d56cfdbf"
May 25 14:16:53.249041 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.248978    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97908028da12dc2f16ec5c48506a0377b167edc4063682beb82c9708d56cfdbf"} err="failed to get container status \"97908028da12dc2f16ec5c48506a0377b167edc4063682beb82c9708d56cfdbf\": rpc error: code = NotFound desc = could not find container \"97908028da12dc2f16ec5c48506a0377b167edc4063682beb82c9708d56cfdbf\": container with ID starting with 97908028da12dc2f16ec5c48506a0377b167edc4063682beb82c9708d56cfdbf not found: ID does not exist"
May 25 14:16:53.249041 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.249001    2571 scope.go:117] "RemoveContainer" containerID="dad6bec88295526c8f300e33fb68ff33892f6877145067fd7cac096d03ca4103"
May 25 14:16:53.256552 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.256535    2571 scope.go:117] "RemoveContainer" containerID="dad6bec88295526c8f300e33fb68ff33892f6877145067fd7cac096d03ca4103"
May 25 14:16:53.256800 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:16:53.256783    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dad6bec88295526c8f300e33fb68ff33892f6877145067fd7cac096d03ca4103\": container with ID starting with dad6bec88295526c8f300e33fb68ff33892f6877145067fd7cac096d03ca4103 not found: ID does not exist" containerID="dad6bec88295526c8f300e33fb68ff33892f6877145067fd7cac096d03ca4103"
May 25 14:16:53.256847 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.256806    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dad6bec88295526c8f300e33fb68ff33892f6877145067fd7cac096d03ca4103"} err="failed to get container status \"dad6bec88295526c8f300e33fb68ff33892f6877145067fd7cac096d03ca4103\": rpc error: code = NotFound desc = could not find container \"dad6bec88295526c8f300e33fb68ff33892f6877145067fd7cac096d03ca4103\": container with ID starting with dad6bec88295526c8f300e33fb68ff33892f6877145067fd7cac096d03ca4103 not found: ID does not exist"
May 25 14:16:53.261785 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.261764    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-854544c579-b28rf"]
May 25 14:16:53.265726 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.265706    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-854544c579-b28rf"]
May 25 14:16:53.274347 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.274328    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-rfpfg"]
May 25 14:16:53.275511 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.275492    2571 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/e5cc0f98-b50d-43e8-a15e-cebdee5e8470-postgresdata\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:16:53.275585 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.275517    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jww6f\" (UniqueName: \"kubernetes.io/projected/61bfe916-4e23-4d41-b201-d838dc778ad1-kube-api-access-jww6f\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:16:53.275585 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.275532    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txqxd\" (UniqueName: \"kubernetes.io/projected/e5cc0f98-b50d-43e8-a15e-cebdee5e8470-kube-api-access-txqxd\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:16:53.277081 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:53.277065    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-rfpfg"]
May 25 14:16:54.825796 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:54.825753    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61bfe916-4e23-4d41-b201-d838dc778ad1" path="/var/lib/kubelet/pods/61bfe916-4e23-4d41-b201-d838dc778ad1/volumes"
May 25 14:16:54.826316 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:16:54.826292    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5cc0f98-b50d-43e8-a15e-cebdee5e8470" path="/var/lib/kubelet/pods/e5cc0f98-b50d-43e8-a15e-cebdee5e8470/volumes"
May 25 14:17:00.321201 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:00.321160    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg" podUID="7dd66b89-f173-4ed2-a2c4-9de8466177d7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused"
May 25 14:17:01.993226 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:01.993186    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p" podUID="ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
May 25 14:17:10.320808 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:10.320768    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg" podUID="7dd66b89-f173-4ed2-a2c4-9de8466177d7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused"
May 25 14:17:11.992608 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:11.992569    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p" podUID="ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
May 25 14:17:11.993003 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:11.992666    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"
May 25 14:17:20.320480 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:20.320438    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg" podUID="7dd66b89-f173-4ed2-a2c4-9de8466177d7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused"
May 25 14:17:20.320844 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:20.320566    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"
May 25 14:17:21.331655 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.331623    2571 generic.go:358] "Generic (PLEG): container finished" podID="ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" containerID="cfa319d25dad5e4c3ebd542ee3c059e7340c71bf2dd82577dd5d8c09ada3f0bd" exitCode=137
May 25 14:17:21.332053 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.331691    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p" event={"ID":"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372","Type":"ContainerDied","Data":"cfa319d25dad5e4c3ebd542ee3c059e7340c71bf2dd82577dd5d8c09ada3f0bd"}
May 25 14:17:21.333700 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.333678    2571 generic.go:358] "Generic (PLEG): container finished" podID="7dd66b89-f173-4ed2-a2c4-9de8466177d7" containerID="c815bb8516f4976915b8f526ca7ed804f2e74fe5c69f881e95e44ebb4ae31996" exitCode=137
May 25 14:17:21.333828 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.333734    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg" event={"ID":"7dd66b89-f173-4ed2-a2c4-9de8466177d7","Type":"ContainerDied","Data":"c815bb8516f4976915b8f526ca7ed804f2e74fe5c69f881e95e44ebb4ae31996"}
May 25 14:17:21.745604 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.745578    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"
May 25 14:17:21.761508 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.761491    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"
May 25 14:17:21.793230 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.793208    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7dd66b89-f173-4ed2-a2c4-9de8466177d7-online-tls\") pod \"7dd66b89-f173-4ed2-a2c4-9de8466177d7\" (UID: \"7dd66b89-f173-4ed2-a2c4-9de8466177d7\") "
May 25 14:17:21.793374 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.793260    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7dd66b89-f173-4ed2-a2c4-9de8466177d7-feast-data\") pod \"7dd66b89-f173-4ed2-a2c4-9de8466177d7\" (UID: \"7dd66b89-f173-4ed2-a2c4-9de8466177d7\") "
May 25 14:17:21.793374 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.793287    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7dd66b89-f173-4ed2-a2c4-9de8466177d7-registry-tls\") pod \"7dd66b89-f173-4ed2-a2c4-9de8466177d7\" (UID: \"7dd66b89-f173-4ed2-a2c4-9de8466177d7\") "
May 25 14:17:21.793374 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.793320    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8nq2\" (UniqueName: \"kubernetes.io/projected/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-kube-api-access-s8nq2\") pod \"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372\" (UID: \"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372\") "
May 25 14:17:21.793374 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.793346    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-feast-data\") pod \"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372\" (UID: \"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372\") "
May 25 14:17:21.793573 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.793388    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-registry-tls\") pod \"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372\" (UID: \"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372\") "
May 25 14:17:21.793573 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.793428    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-online-tls\") pod \"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372\" (UID: \"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372\") "
May 25 14:17:21.793573 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.793477    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtxk5\" (UniqueName: \"kubernetes.io/projected/7dd66b89-f173-4ed2-a2c4-9de8466177d7-kube-api-access-jtxk5\") pod \"7dd66b89-f173-4ed2-a2c4-9de8466177d7\" (UID: \"7dd66b89-f173-4ed2-a2c4-9de8466177d7\") "
May 25 14:17:21.793729 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.793707    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dd66b89-f173-4ed2-a2c4-9de8466177d7-feast-data" (OuterVolumeSpecName: "feast-data") pod "7dd66b89-f173-4ed2-a2c4-9de8466177d7" (UID: "7dd66b89-f173-4ed2-a2c4-9de8466177d7"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 14:17:21.796191 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.796156    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7dd66b89-f173-4ed2-a2c4-9de8466177d7-online-tls" (OuterVolumeSpecName: "online-tls") pod "7dd66b89-f173-4ed2-a2c4-9de8466177d7" (UID: "7dd66b89-f173-4ed2-a2c4-9de8466177d7"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:17:21.796456 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.796433    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-online-tls" (OuterVolumeSpecName: "online-tls") pod "ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" (UID: "ba4fcc3a-c3f6-478d-b2d3-079be6b3d372"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:17:21.796539 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.796448    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dd66b89-f173-4ed2-a2c4-9de8466177d7-kube-api-access-jtxk5" (OuterVolumeSpecName: "kube-api-access-jtxk5") pod "7dd66b89-f173-4ed2-a2c4-9de8466177d7" (UID: "7dd66b89-f173-4ed2-a2c4-9de8466177d7"). InnerVolumeSpecName "kube-api-access-jtxk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 14:17:21.796539 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.796470    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-kube-api-access-s8nq2" (OuterVolumeSpecName: "kube-api-access-s8nq2") pod "ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" (UID: "ba4fcc3a-c3f6-478d-b2d3-079be6b3d372"). InnerVolumeSpecName "kube-api-access-s8nq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 14:17:21.796539 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.796521    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" (UID: "ba4fcc3a-c3f6-478d-b2d3-079be6b3d372"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:17:21.796691 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.796539    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7dd66b89-f173-4ed2-a2c4-9de8466177d7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7dd66b89-f173-4ed2-a2c4-9de8466177d7" (UID: "7dd66b89-f173-4ed2-a2c4-9de8466177d7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:17:21.803530 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.803504    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-feast-data" (OuterVolumeSpecName: "feast-data") pod "ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" (UID: "ba4fcc3a-c3f6-478d-b2d3-079be6b3d372"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 14:17:21.894364 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.894303    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8nq2\" (UniqueName: \"kubernetes.io/projected/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-kube-api-access-s8nq2\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:17:21.894364 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.894327    2571 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-feast-data\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:17:21.894364 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.894336    2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-registry-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:17:21.894364 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.894345    2571 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372-online-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:17:21.894364 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.894354    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtxk5\" (UniqueName: \"kubernetes.io/projected/7dd66b89-f173-4ed2-a2c4-9de8466177d7-kube-api-access-jtxk5\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:17:21.894364 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.894362    2571 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7dd66b89-f173-4ed2-a2c4-9de8466177d7-online-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:17:21.894364 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.894371    2571 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7dd66b89-f173-4ed2-a2c4-9de8466177d7-feast-data\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:17:21.894717 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:21.894380    2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7dd66b89-f173-4ed2-a2c4-9de8466177d7-registry-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:17:22.343578 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:22.343544    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p" event={"ID":"ba4fcc3a-c3f6-478d-b2d3-079be6b3d372","Type":"ContainerDied","Data":"956ae57cc547c2a79a0e372295ccbb3eae818ebe7fe4b33232de526fb973f261"}
May 25 14:17:22.344019 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:22.343597    2571 scope.go:117] "RemoveContainer" containerID="b287807a323c6835436cfcc26889f29ee11c25f1ed97aab1dee15802cfd4d5f2"
May 25 14:17:22.344019 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:22.343604    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"
May 25 14:17:22.345813 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:22.345793    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg" event={"ID":"7dd66b89-f173-4ed2-a2c4-9de8466177d7","Type":"ContainerDied","Data":"ca70c354e761900b91677834f0102d573640a11e9c2ab65b2b9f3d72cc640ddc"}
May 25 14:17:22.345917 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:22.345852    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"
May 25 14:17:22.352728 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:22.352707    2571 scope.go:117] "RemoveContainer" containerID="cfa319d25dad5e4c3ebd542ee3c059e7340c71bf2dd82577dd5d8c09ada3f0bd"
May 25 14:17:22.360471 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:22.360454    2571 scope.go:117] "RemoveContainer" containerID="278f778c4b6cb97db42653e78a4e2694e5376a44aa24eeabebb10bb6187c4609"
May 25 14:17:22.367537 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:22.367522    2571 scope.go:117] "RemoveContainer" containerID="2e973f678aad835b173e1fcbfc20890ebe2595cd25d1a35b69650367f09021dd"
May 25 14:17:22.371418 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:22.371398    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"]
May 25 14:17:22.375286 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:22.375267    2571 scope.go:117] "RemoveContainer" containerID="62763710ecb049e597a7ead5537d48667dc16eb82cc042615d87b250acdcf7af"
May 25 14:17:22.376010 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:22.375992    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-5c95685c9-r2j9p"]
May 25 14:17:22.383767 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:22.383750    2571 scope.go:117] "RemoveContainer" containerID="c815bb8516f4976915b8f526ca7ed804f2e74fe5c69f881e95e44ebb4ae31996"
May 25 14:17:22.387603 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:22.387582    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"]
May 25 14:17:22.391472 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:22.391453    2571 scope.go:117] "RemoveContainer" containerID="8a6242342de44bafee987ce80f4f00e72304eb1548a798131145a376a200cb67"
May 25 14:17:22.393462 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:22.393442    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7784d54cb8-g75jg"]
May 25 14:17:22.398547 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:22.398529    2571 scope.go:117] "RemoveContainer" containerID="bf8888d593bd68fc4633b2d5d14f95ea8de0bba943b42f258bf0d44d6fa58fda"
May 25 14:17:22.825738 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:22.825703    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7dd66b89-f173-4ed2-a2c4-9de8466177d7" path="/var/lib/kubelet/pods/7dd66b89-f173-4ed2-a2c4-9de8466177d7/volumes"
May 25 14:17:22.826263 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:22.826247    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" path="/var/lib/kubelet/pods/ba4fcc3a-c3f6-478d-b2d3-079be6b3d372/volumes"
May 25 14:17:23.033455 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.033422    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"]
May 25 14:17:23.033914 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.033899    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dd66b89-f173-4ed2-a2c4-9de8466177d7" containerName="feast-apply"
May 25 14:17:23.033998 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.033918    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dd66b89-f173-4ed2-a2c4-9de8466177d7" containerName="feast-apply"
May 25 14:17:23.033998 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.033952    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5cc0f98-b50d-43e8-a15e-cebdee5e8470" containerName="postgres"
May 25 14:17:23.033998 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.033962    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5cc0f98-b50d-43e8-a15e-cebdee5e8470" containerName="postgres"
May 25 14:17:23.033998 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.033981    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" containerName="feast-apply"
May 25 14:17:23.033998 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.033990    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" containerName="feast-apply"
May 25 14:17:23.033998 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.033998    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" containerName="online"
May 25 14:17:23.034209 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.034006    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" containerName="online"
May 25 14:17:23.034209 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.034015    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dd66b89-f173-4ed2-a2c4-9de8466177d7" containerName="feast-init"
May 25 14:17:23.034209 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.034023    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dd66b89-f173-4ed2-a2c4-9de8466177d7" containerName="feast-init"
May 25 14:17:23.034209 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.034034    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" containerName="registry"
May 25 14:17:23.034209 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.034042    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" containerName="registry"
May 25 14:17:23.034209 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.034055    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" containerName="feast-init"
May 25 14:17:23.034209 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.034063    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" containerName="feast-init"
May 25 14:17:23.034209 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.034072    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dd66b89-f173-4ed2-a2c4-9de8466177d7" containerName="registry"
May 25 14:17:23.034209 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.034080    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dd66b89-f173-4ed2-a2c4-9de8466177d7" containerName="registry"
May 25 14:17:23.034209 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.034093    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dd66b89-f173-4ed2-a2c4-9de8466177d7" containerName="online"
May 25 14:17:23.034209 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.034103    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dd66b89-f173-4ed2-a2c4-9de8466177d7" containerName="online"
May 25 14:17:23.034209 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.034117    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61bfe916-4e23-4d41-b201-d838dc778ad1" containerName="redis"
May 25 14:17:23.034209 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.034125    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="61bfe916-4e23-4d41-b201-d838dc778ad1" containerName="redis"
May 25 14:17:23.034209 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.034201    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="e5cc0f98-b50d-43e8-a15e-cebdee5e8470" containerName="postgres"
May 25 14:17:23.034209 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.034213    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" containerName="online"
May 25 14:17:23.034690 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.034225    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="7dd66b89-f173-4ed2-a2c4-9de8466177d7" containerName="online"
May 25 14:17:23.034690 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.034236    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="ba4fcc3a-c3f6-478d-b2d3-079be6b3d372" containerName="registry"
May 25 14:17:23.034690 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.034247    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="7dd66b89-f173-4ed2-a2c4-9de8466177d7" containerName="registry"
May 25 14:17:23.034690 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.034255    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="61bfe916-4e23-4d41-b201-d838dc778ad1" containerName="redis"
May 25 14:17:23.040091 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.040071    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:23.042397 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.042374    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 25 14:17:23.042522 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.042422    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 25 14:17:23.042522 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.042459    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 25 14:17:23.042522 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.042463    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-h8m7n\""
May 25 14:17:23.042522 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.042483    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 25 14:17:23.042754 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.042729    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 25 14:17:23.042754 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.042750    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 25 14:17:23.050263 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.050238    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"]
May 25 14:17:23.104619 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.104540    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-sflgt\" (UID: \"a4c9418d-f5e2-4651-936d-88c4297319b5\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:23.104619 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.104590    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-sflgt\" (UID: \"a4c9418d-f5e2-4651-936d-88c4297319b5\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:23.104792 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.104656    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a4c9418d-f5e2-4651-936d-88c4297319b5-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-sflgt\" (UID: \"a4c9418d-f5e2-4651-936d-88c4297319b5\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:23.104792 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.104687    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-sflgt\" (UID: \"a4c9418d-f5e2-4651-936d-88c4297319b5\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:23.104792 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.104733    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sztx9\" (UniqueName: \"kubernetes.io/projected/a4c9418d-f5e2-4651-936d-88c4297319b5-kube-api-access-sztx9\") pod \"feast-simple-feast-setup-664fcfcc58-sflgt\" (UID: \"a4c9418d-f5e2-4651-936d-88c4297319b5\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:23.104792 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.104784    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-sflgt\" (UID: \"a4c9418d-f5e2-4651-936d-88c4297319b5\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:23.205911 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.205887    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-sflgt\" (UID: \"a4c9418d-f5e2-4651-936d-88c4297319b5\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:23.206061 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.205925    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-sflgt\" (UID: \"a4c9418d-f5e2-4651-936d-88c4297319b5\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:23.206061 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.205986    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a4c9418d-f5e2-4651-936d-88c4297319b5-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-sflgt\" (UID: \"a4c9418d-f5e2-4651-936d-88c4297319b5\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:23.206061 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.206010    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-sflgt\" (UID: \"a4c9418d-f5e2-4651-936d-88c4297319b5\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:23.206061 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.206052    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sztx9\" (UniqueName: \"kubernetes.io/projected/a4c9418d-f5e2-4651-936d-88c4297319b5-kube-api-access-sztx9\") pod \"feast-simple-feast-setup-664fcfcc58-sflgt\" (UID: \"a4c9418d-f5e2-4651-936d-88c4297319b5\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:23.206257 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.206098    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-sflgt\" (UID: \"a4c9418d-f5e2-4651-936d-88c4297319b5\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:23.206257 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:17:23.206120    2571 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found
May 25 14:17:23.206257 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:17:23.206190    2571 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 25 14:17:23.206257 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:17:23.206205    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-online-tls podName:a4c9418d-f5e2-4651-936d-88c4297319b5 nodeName:}" failed. No retries permitted until 2026-05-25 14:17:23.706183922 +0000 UTC m=+1051.541039992 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-online-tls") pod "feast-simple-feast-setup-664fcfcc58-sflgt" (UID: "a4c9418d-f5e2-4651-936d-88c4297319b5") : secret "feast-simple-feast-setup-online-tls" not found
May 25 14:17:23.206257 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:17:23.206250    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-ui-tls podName:a4c9418d-f5e2-4651-936d-88c4297319b5 nodeName:}" failed. No retries permitted until 2026-05-25 14:17:23.706232358 +0000 UTC m=+1051.541088431 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-ui-tls") pod "feast-simple-feast-setup-664fcfcc58-sflgt" (UID: "a4c9418d-f5e2-4651-936d-88c4297319b5") : secret "feast-simple-feast-setup-ui-tls" not found
May 25 14:17:23.206528 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.206444    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a4c9418d-f5e2-4651-936d-88c4297319b5-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-sflgt\" (UID: \"a4c9418d-f5e2-4651-936d-88c4297319b5\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:23.208527 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.208505    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-sflgt\" (UID: \"a4c9418d-f5e2-4651-936d-88c4297319b5\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:23.208617 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.208546    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-sflgt\" (UID: \"a4c9418d-f5e2-4651-936d-88c4297319b5\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:23.215041 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.215016    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sztx9\" (UniqueName: \"kubernetes.io/projected/a4c9418d-f5e2-4651-936d-88c4297319b5-kube-api-access-sztx9\") pod \"feast-simple-feast-setup-664fcfcc58-sflgt\" (UID: \"a4c9418d-f5e2-4651-936d-88c4297319b5\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:23.710755 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.710718    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-sflgt\" (UID: \"a4c9418d-f5e2-4651-936d-88c4297319b5\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:23.711164 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.710779    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-sflgt\" (UID: \"a4c9418d-f5e2-4651-936d-88c4297319b5\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:23.713296 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.713271    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-sflgt\" (UID: \"a4c9418d-f5e2-4651-936d-88c4297319b5\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:23.713404 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.713300    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-sflgt\" (UID: \"a4c9418d-f5e2-4651-936d-88c4297319b5\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:23.959155 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:23.959130    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:24.092886 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:24.092856    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"]
May 25 14:17:24.095269 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:17:24.095242    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4c9418d_f5e2_4651_936d_88c4297319b5.slice/crio-611213fca5b3dd78691181ca9fe0ddd5401109d7d5587f4cce706219e0d2403e WatchSource:0}: Error finding container 611213fca5b3dd78691181ca9fe0ddd5401109d7d5587f4cce706219e0d2403e: Status 404 returned error can't find the container with id 611213fca5b3dd78691181ca9fe0ddd5401109d7d5587f4cce706219e0d2403e
May 25 14:17:24.097070 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:24.097053    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 25 14:17:24.353777 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:24.353699    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" event={"ID":"a4c9418d-f5e2-4651-936d-88c4297319b5","Type":"ContainerStarted","Data":"611213fca5b3dd78691181ca9fe0ddd5401109d7d5587f4cce706219e0d2403e"}
May 25 14:17:38.406966 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:38.406916    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" event={"ID":"a4c9418d-f5e2-4651-936d-88c4297319b5","Type":"ContainerStarted","Data":"12870144bac56cd36c14d1fc76affa38d1a188936b81f0bcbd5d3d8d6941b3a7"}
May 25 14:17:43.429910 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:43.429805    2571 generic.go:358] "Generic (PLEG): container finished" podID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerID="12870144bac56cd36c14d1fc76affa38d1a188936b81f0bcbd5d3d8d6941b3a7" exitCode=0
May 25 14:17:43.429910 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:43.429878    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" event={"ID":"a4c9418d-f5e2-4651-936d-88c4297319b5","Type":"ContainerDied","Data":"12870144bac56cd36c14d1fc76affa38d1a188936b81f0bcbd5d3d8d6941b3a7"}
May 25 14:17:44.436306 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:44.436261    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" event={"ID":"a4c9418d-f5e2-4651-936d-88c4297319b5","Type":"ContainerStarted","Data":"e564b1cffcb65672bf3da075ae6a091bcf775ab3907833abf2029e4284e7cc4e"}
May 25 14:17:50.460500 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:50.460459    2571 generic.go:358] "Generic (PLEG): container finished" podID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerID="e564b1cffcb65672bf3da075ae6a091bcf775ab3907833abf2029e4284e7cc4e" exitCode=0
May 25 14:17:50.460926 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:50.460537    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" event={"ID":"a4c9418d-f5e2-4651-936d-88c4297319b5","Type":"ContainerDied","Data":"e564b1cffcb65672bf3da075ae6a091bcf775ab3907833abf2029e4284e7cc4e"}
May 25 14:17:51.470523 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:51.470473    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" event={"ID":"a4c9418d-f5e2-4651-936d-88c4297319b5","Type":"ContainerStarted","Data":"46c3f9bc7d357a3e240a4624d70ae94150d08db48d28363dec8a2fcdd1e1fedd"}
May 25 14:17:51.471128 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:51.470532    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" event={"ID":"a4c9418d-f5e2-4651-936d-88c4297319b5","Type":"ContainerStarted","Data":"8f716da4084f2fe1a0f0dc200ef3d62143e2f9b50cddbcd9d49a29c0cbf81e4f"}
May 25 14:17:51.471128 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:51.470551    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" event={"ID":"a4c9418d-f5e2-4651-936d-88c4297319b5","Type":"ContainerStarted","Data":"75cefc9f835014b222f353abd452a99fc3de9400c7c32d575a350d2a5dd7a68d"}
May 25 14:17:51.471128 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:51.470565    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" event={"ID":"a4c9418d-f5e2-4651-936d-88c4297319b5","Type":"ContainerStarted","Data":"4a395820bf5c086fc0b0c232c6d6ad908acc2504171cbf7876f9531b11dc5d91"}
May 25 14:17:51.498732 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:51.497850    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" podStartSLOduration=14.301597775 podStartE2EDuration="28.497797413s" podCreationTimestamp="2026-05-25 14:17:23 +0000 UTC" firstStartedPulling="2026-05-25 14:17:24.097210309 +0000 UTC m=+1051.932066383" lastFinishedPulling="2026-05-25 14:17:38.293409951 +0000 UTC m=+1066.128266021" observedRunningTime="2026-05-25 14:17:51.495239393 +0000 UTC m=+1079.330095500" watchObservedRunningTime="2026-05-25 14:17:51.497797413 +0000 UTC m=+1079.332653506"
May 25 14:17:53.959548 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:53.959504    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:53.960242 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:53.960216    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:53.960360 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:53.960349    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:53.960753 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:53.960739    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:53.960860 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:53.960849    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:53.960986 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:53.960974    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:53.961095 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:53.961083    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:53.961183 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:53.961173    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:53.962193 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:53.962030    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.34:8816: connect: connection refused"
May 25 14:17:53.962193 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:53.962047    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.34:8443: connect: connection refused"
May 25 14:17:53.962193 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:53.962063    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.34:6571: connect: connection refused"
May 25 14:17:53.962193 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:53.962117    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="online" probeResult="failure" output="Get \"https://10.134.0.34:6567/health\": dial tcp 10.134.0.34:6567: connect: connection refused"
May 25 14:17:56.960348 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:56.960291    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.34:8443: connect: connection refused"
May 25 14:17:56.960855 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:56.960291    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.34:8816: connect: connection refused"
May 25 14:17:56.960855 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:56.960530    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="online" probeResult="failure" output="Get \"https://10.134.0.34:6567/health\": dial tcp 10.134.0.34:6567: connect: connection refused"
May 25 14:17:56.961043 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:56.961014    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.34:6571: connect: connection refused"
May 25 14:17:59.960917 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:59.960884    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:59.961353 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:59.961054    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:59.961353 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:59.961319    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:59.962106 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:59.962081    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:59.962364 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:59.962341    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:59.962459 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:59.962388    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:17:59.965977 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:17:59.965950    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:18:00.509155 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:18:00.509120    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:19:02.434308 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:02.434208    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"]
May 25 14:19:02.434911 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:02.434610    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="registry" containerID="cri-o://4a395820bf5c086fc0b0c232c6d6ad908acc2504171cbf7876f9531b11dc5d91" gracePeriod=30
May 25 14:19:02.434911 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:02.434682    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="offline" containerID="cri-o://8f716da4084f2fe1a0f0dc200ef3d62143e2f9b50cddbcd9d49a29c0cbf81e4f" gracePeriod=30
May 25 14:19:02.434911 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:02.434677    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="ui" containerID="cri-o://46c3f9bc7d357a3e240a4624d70ae94150d08db48d28363dec8a2fcdd1e1fedd" gracePeriod=30
May 25 14:19:02.434911 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:02.434768    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="online" containerID="cri-o://75cefc9f835014b222f353abd452a99fc3de9400c7c32d575a350d2a5dd7a68d" gracePeriod=30
May 25 14:19:02.840251 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:02.840206    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"]
May 25 14:19:02.845045 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:02.845005    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:02.847283 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:02.847243    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-r4jdw\""
May 25 14:19:02.859226 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:02.859176    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"]
May 25 14:19:02.975553 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:02.975490    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-9q5jm\" (UID: \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:02.975553 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:02.975561    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-9q5jm\" (UID: \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:02.975840 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:02.975636    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-475dc\" (UniqueName: \"kubernetes.io/projected/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-kube-api-access-475dc\") pod \"feast-simple-feast-setup-664fcfcc58-9q5jm\" (UID: \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:02.975840 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:02.975664    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-9q5jm\" (UID: \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:02.975840 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:02.975723    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-9q5jm\" (UID: \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:02.975840 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:02.975790    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-9q5jm\" (UID: \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:03.077171 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:03.077124    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-9q5jm\" (UID: \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:03.077171 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:03.077172    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-9q5jm\" (UID: \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:03.077457 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:03.077203    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-9q5jm\" (UID: \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:03.077457 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:03.077226    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-9q5jm\" (UID: \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:03.077457 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:03.077279    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-475dc\" (UniqueName: \"kubernetes.io/projected/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-kube-api-access-475dc\") pod \"feast-simple-feast-setup-664fcfcc58-9q5jm\" (UID: \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:03.077907 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:03.077726    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-9q5jm\" (UID: \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:03.077907 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:03.077773    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-9q5jm\" (UID: \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:03.080300 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:03.080220    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-9q5jm\" (UID: \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:03.080469 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:03.080372    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-9q5jm\" (UID: \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:03.081028 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:03.081005    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-9q5jm\" (UID: \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:03.081172 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:03.081098    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-9q5jm\" (UID: \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:03.085769 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:03.085734    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-475dc\" (UniqueName: \"kubernetes.io/projected/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-kube-api-access-475dc\") pod \"feast-simple-feast-setup-664fcfcc58-9q5jm\" (UID: \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:03.160072 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:03.159960    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:03.320367 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:03.320336    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"]
May 25 14:19:03.323456 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:19:03.323421    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod87cfdfae_aa95_4b1d_8f0d_f9c76021b9ad.slice/crio-847380af4ee1823bf4b39b9dbcd4c9715d0f2881d6c9969b03922f8b4159d23d WatchSource:0}: Error finding container 847380af4ee1823bf4b39b9dbcd4c9715d0f2881d6c9969b03922f8b4159d23d: Status 404 returned error can't find the container with id 847380af4ee1823bf4b39b9dbcd4c9715d0f2881d6c9969b03922f8b4159d23d
May 25 14:19:03.724533 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:03.724485    2571 generic.go:358] "Generic (PLEG): container finished" podID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerID="46c3f9bc7d357a3e240a4624d70ae94150d08db48d28363dec8a2fcdd1e1fedd" exitCode=0
May 25 14:19:03.724533 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:03.724517    2571 generic.go:358] "Generic (PLEG): container finished" podID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerID="75cefc9f835014b222f353abd452a99fc3de9400c7c32d575a350d2a5dd7a68d" exitCode=0
May 25 14:19:03.725085 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:03.724562    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" event={"ID":"a4c9418d-f5e2-4651-936d-88c4297319b5","Type":"ContainerDied","Data":"46c3f9bc7d357a3e240a4624d70ae94150d08db48d28363dec8a2fcdd1e1fedd"}
May 25 14:19:03.725085 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:03.724605    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" event={"ID":"a4c9418d-f5e2-4651-936d-88c4297319b5","Type":"ContainerDied","Data":"75cefc9f835014b222f353abd452a99fc3de9400c7c32d575a350d2a5dd7a68d"}
May 25 14:19:03.726227 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:03.726188    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" event={"ID":"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad","Type":"ContainerStarted","Data":"ddeea4e535e5eec2b189dc8e26c65dfd28a53f4b4a2ac8892b88160ca21ec209"}
May 25 14:19:03.726336 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:03.726234    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" event={"ID":"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad","Type":"ContainerStarted","Data":"847380af4ee1823bf4b39b9dbcd4c9715d0f2881d6c9969b03922f8b4159d23d"}
May 25 14:19:09.747797 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:09.747756    2571 generic.go:358] "Generic (PLEG): container finished" podID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerID="ddeea4e535e5eec2b189dc8e26c65dfd28a53f4b4a2ac8892b88160ca21ec209" exitCode=0
May 25 14:19:09.748414 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:09.747845    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" event={"ID":"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad","Type":"ContainerDied","Data":"ddeea4e535e5eec2b189dc8e26c65dfd28a53f4b4a2ac8892b88160ca21ec209"}
May 25 14:19:09.962102 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:09.962050    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.34:8443: connect: connection refused"
May 25 14:19:10.506308 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:10.506258    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="online" probeResult="failure" output="Get \"https://10.134.0.34:6567/health\": dial tcp 10.134.0.34:6567: connect: connection refused"
May 25 14:19:10.754047 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:10.754006    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" event={"ID":"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad","Type":"ContainerStarted","Data":"aa56a6cc877dd6da02d56663347d9f23651542367184f55efb460f868c1be6b9"}
May 25 14:19:16.776072 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:16.776026    2571 generic.go:358] "Generic (PLEG): container finished" podID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerID="aa56a6cc877dd6da02d56663347d9f23651542367184f55efb460f868c1be6b9" exitCode=0
May 25 14:19:16.776524 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:16.776097    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" event={"ID":"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad","Type":"ContainerDied","Data":"aa56a6cc877dd6da02d56663347d9f23651542367184f55efb460f868c1be6b9"}
May 25 14:19:17.785124 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:17.785071    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" event={"ID":"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad","Type":"ContainerStarted","Data":"ac4e6b36648b38fbcd0464297b928ce01d50bb688519b22d3da752d7de6f0104"}
May 25 14:19:17.785662 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:17.785137    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" event={"ID":"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad","Type":"ContainerStarted","Data":"a58e415ff51ef4e9ebc494f97895d2df5997aea6d3bdd5139af6dbfe200241d0"}
May 25 14:19:17.785662 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:17.785158    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" event={"ID":"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad","Type":"ContainerStarted","Data":"49ff8a6e57f5bc8aa65c37813ad0723cef60e221e5596d16f56dba7a761ad461"}
May 25 14:19:17.785662 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:17.785172    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" event={"ID":"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad","Type":"ContainerStarted","Data":"e19c49fa409d84064f36be2ff44eb8609caac683b0948bfae34b0f9c75e34d4a"}
May 25 14:19:17.811091 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:17.811018    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" podStartSLOduration=15.810992216 podStartE2EDuration="15.810992216s" podCreationTimestamp="2026-05-25 14:19:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 14:19:17.808511981 +0000 UTC m=+1165.643368075" watchObservedRunningTime="2026-05-25 14:19:17.810992216 +0000 UTC m=+1165.645848313"
May 25 14:19:18.161071 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:18.160963    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:18.161249 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:18.161017    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:18.161249 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:18.161207    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:18.161249 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:18.161223    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:18.163800 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:18.163382    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused"
May 25 14:19:18.163800 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:18.163412    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.35:8816: connect: connection refused"
May 25 14:19:18.163800 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:18.163446    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
May 25 14:19:18.163800 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:18.163493    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.35:6571: connect: connection refused"
May 25 14:19:19.962433 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:19.962383    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.34:8443: connect: connection refused"
May 25 14:19:20.506568 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:20.506416    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="online" probeResult="failure" output="Get \"https://10.134.0.34:6567/health\": dial tcp 10.134.0.34:6567: connect: connection refused"
May 25 14:19:21.161444 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:21.161353    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused"
May 25 14:19:21.162104 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:21.161353    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.35:8816: connect: connection refused"
May 25 14:19:21.162104 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:21.161365    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
May 25 14:19:21.162104 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:21.161353    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.35:6571: connect: connection refused"
May 25 14:19:23.160877 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:23.160825    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:23.161674 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:23.161640    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:23.161830 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:23.161818    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:23.162228 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:23.162207    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:24.161445 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:24.161392    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
May 25 14:19:24.162033 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:24.161954    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.35:6571: connect: connection refused"
May 25 14:19:24.162125 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:24.162035    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused"
May 25 14:19:24.162192 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:24.162144    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.35:8816: connect: connection refused"
May 25 14:19:27.161227 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:27.161189    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:27.161720 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:27.161694    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:27.161866 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:27.161852    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:27.162611 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:27.162580    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:27.162986 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:27.162962    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:27.163131 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:27.163115    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:27.168656 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:27.168616    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:27.832743 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:27.832703    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:19:29.962144 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:29.962094    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.34:8443: connect: connection refused"
May 25 14:19:29.962577 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:29.962236    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:19:30.506957 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:30.506871    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="online" probeResult="failure" output="Get \"https://10.134.0.34:6567/health\": dial tcp 10.134.0.34:6567: connect: connection refused"
May 25 14:19:30.507185 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:30.507044    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:19:32.854980 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:32.854869    2571 generic.go:358] "Generic (PLEG): container finished" podID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerID="8f716da4084f2fe1a0f0dc200ef3d62143e2f9b50cddbcd9d49a29c0cbf81e4f" exitCode=137
May 25 14:19:32.854980 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:32.854896    2571 generic.go:358] "Generic (PLEG): container finished" podID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerID="4a395820bf5c086fc0b0c232c6d6ad908acc2504171cbf7876f9531b11dc5d91" exitCode=137
May 25 14:19:32.854980 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:32.854969    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" event={"ID":"a4c9418d-f5e2-4651-936d-88c4297319b5","Type":"ContainerDied","Data":"8f716da4084f2fe1a0f0dc200ef3d62143e2f9b50cddbcd9d49a29c0cbf81e4f"}
May 25 14:19:32.855428 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:32.855006    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" event={"ID":"a4c9418d-f5e2-4651-936d-88c4297319b5","Type":"ContainerDied","Data":"4a395820bf5c086fc0b0c232c6d6ad908acc2504171cbf7876f9531b11dc5d91"}
May 25 14:19:33.104575 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.104542    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:19:33.172211 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.172109    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sztx9\" (UniqueName: \"kubernetes.io/projected/a4c9418d-f5e2-4651-936d-88c4297319b5-kube-api-access-sztx9\") pod \"a4c9418d-f5e2-4651-936d-88c4297319b5\" (UID: \"a4c9418d-f5e2-4651-936d-88c4297319b5\") "
May 25 14:19:33.172376 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.172246    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-online-tls\") pod \"a4c9418d-f5e2-4651-936d-88c4297319b5\" (UID: \"a4c9418d-f5e2-4651-936d-88c4297319b5\") "
May 25 14:19:33.172376 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.172276    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a4c9418d-f5e2-4651-936d-88c4297319b5-feast-data\") pod \"a4c9418d-f5e2-4651-936d-88c4297319b5\" (UID: \"a4c9418d-f5e2-4651-936d-88c4297319b5\") "
May 25 14:19:33.172376 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.172303    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-offline-tls\") pod \"a4c9418d-f5e2-4651-936d-88c4297319b5\" (UID: \"a4c9418d-f5e2-4651-936d-88c4297319b5\") "
May 25 14:19:33.172376 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.172329    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-ui-tls\") pod \"a4c9418d-f5e2-4651-936d-88c4297319b5\" (UID: \"a4c9418d-f5e2-4651-936d-88c4297319b5\") "
May 25 14:19:33.172376 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.172374    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-registry-tls\") pod \"a4c9418d-f5e2-4651-936d-88c4297319b5\" (UID: \"a4c9418d-f5e2-4651-936d-88c4297319b5\") "
May 25 14:19:33.172883 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.172848    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4c9418d-f5e2-4651-936d-88c4297319b5-feast-data" (OuterVolumeSpecName: "feast-data") pod "a4c9418d-f5e2-4651-936d-88c4297319b5" (UID: "a4c9418d-f5e2-4651-936d-88c4297319b5"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 14:19:33.175095 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.175048    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "a4c9418d-f5e2-4651-936d-88c4297319b5" (UID: "a4c9418d-f5e2-4651-936d-88c4297319b5"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:19:33.175324 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.175241    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a4c9418d-f5e2-4651-936d-88c4297319b5" (UID: "a4c9418d-f5e2-4651-936d-88c4297319b5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:19:33.175324 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.175280    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4c9418d-f5e2-4651-936d-88c4297319b5-kube-api-access-sztx9" (OuterVolumeSpecName: "kube-api-access-sztx9") pod "a4c9418d-f5e2-4651-936d-88c4297319b5" (UID: "a4c9418d-f5e2-4651-936d-88c4297319b5"). InnerVolumeSpecName "kube-api-access-sztx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 14:19:33.175324 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.175274    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-online-tls" (OuterVolumeSpecName: "online-tls") pod "a4c9418d-f5e2-4651-936d-88c4297319b5" (UID: "a4c9418d-f5e2-4651-936d-88c4297319b5"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:19:33.175324 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.175292    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "a4c9418d-f5e2-4651-936d-88c4297319b5" (UID: "a4c9418d-f5e2-4651-936d-88c4297319b5"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:19:33.273833 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.273779    2571 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-offline-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:19:33.273833 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.273817    2571 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-ui-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:19:33.273833 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.273830    2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-registry-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:19:33.273833 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.273845    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sztx9\" (UniqueName: \"kubernetes.io/projected/a4c9418d-f5e2-4651-936d-88c4297319b5-kube-api-access-sztx9\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:19:33.274196 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.273856    2571 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a4c9418d-f5e2-4651-936d-88c4297319b5-online-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:19:33.274196 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.273865    2571 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a4c9418d-f5e2-4651-936d-88c4297319b5-feast-data\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:19:33.860778 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.860736    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt" event={"ID":"a4c9418d-f5e2-4651-936d-88c4297319b5","Type":"ContainerDied","Data":"611213fca5b3dd78691181ca9fe0ddd5401109d7d5587f4cce706219e0d2403e"}
May 25 14:19:33.860778 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.860788    2571 scope.go:117] "RemoveContainer" containerID="46c3f9bc7d357a3e240a4624d70ae94150d08db48d28363dec8a2fcdd1e1fedd"
May 25 14:19:33.861330 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.860863    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"
May 25 14:19:33.871577 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.871552    2571 scope.go:117] "RemoveContainer" containerID="8f716da4084f2fe1a0f0dc200ef3d62143e2f9b50cddbcd9d49a29c0cbf81e4f"
May 25 14:19:33.881048 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.881025    2571 scope.go:117] "RemoveContainer" containerID="75cefc9f835014b222f353abd452a99fc3de9400c7c32d575a350d2a5dd7a68d"
May 25 14:19:33.884168 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.884132    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"]
May 25 14:19:33.887833 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.887794    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-sflgt"]
May 25 14:19:33.891607 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.891586    2571 scope.go:117] "RemoveContainer" containerID="4a395820bf5c086fc0b0c232c6d6ad908acc2504171cbf7876f9531b11dc5d91"
May 25 14:19:33.900780 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.900759    2571 scope.go:117] "RemoveContainer" containerID="e564b1cffcb65672bf3da075ae6a091bcf775ab3907833abf2029e4284e7cc4e"
May 25 14:19:33.909952 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:33.909910    2571 scope.go:117] "RemoveContainer" containerID="12870144bac56cd36c14d1fc76affa38d1a188936b81f0bcbd5d3d8d6941b3a7"
May 25 14:19:34.826761 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:19:34.826721    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" path="/var/lib/kubelet/pods/a4c9418d-f5e2-4651-936d-88c4297319b5/volumes"
May 25 14:20:30.668741 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.668693    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"]
May 25 14:20:30.669350 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.669144    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="online"
May 25 14:20:30.669350 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.669162    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="online"
May 25 14:20:30.669350 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.669175    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="offline"
May 25 14:20:30.669350 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.669181    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="offline"
May 25 14:20:30.669350 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.669193    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="registry"
May 25 14:20:30.669350 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.669198    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="registry"
May 25 14:20:30.669350 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.669207    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="feast-init"
May 25 14:20:30.669350 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.669212    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="feast-init"
May 25 14:20:30.669350 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.669219    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="ui"
May 25 14:20:30.669350 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.669224    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="ui"
May 25 14:20:30.669350 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.669236    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="feast-apply"
May 25 14:20:30.669350 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.669245    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="feast-apply"
May 25 14:20:30.669350 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.669309    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="ui"
May 25 14:20:30.669350 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.669318    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="online"
May 25 14:20:30.669350 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.669331    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="registry"
May 25 14:20:30.669350 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.669342    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="a4c9418d-f5e2-4651-936d-88c4297319b5" containerName="offline"
May 25 14:20:30.672974 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.672923    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:30.675312 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.675271    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 25 14:20:30.675312 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.675298    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 25 14:20:30.675552 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.675303    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 25 14:20:30.675552 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.675271    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 25 14:20:30.675552 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.675448    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 25 14:20:30.675680 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.675670    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 25 14:20:30.675768 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.675754    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-kqht4\""
May 25 14:20:30.684806 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.684771    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"]
May 25 14:20:30.720379 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.720332    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/053647d8-73f9-462d-97cd-cc771937306d-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmhgn\" (UID: \"053647d8-73f9-462d-97cd-cc771937306d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:30.720379 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.720383    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/053647d8-73f9-462d-97cd-cc771937306d-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmhgn\" (UID: \"053647d8-73f9-462d-97cd-cc771937306d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:30.720605 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.720443    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/053647d8-73f9-462d-97cd-cc771937306d-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmhgn\" (UID: \"053647d8-73f9-462d-97cd-cc771937306d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:30.720605 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.720527    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/053647d8-73f9-462d-97cd-cc771937306d-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmhgn\" (UID: \"053647d8-73f9-462d-97cd-cc771937306d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:30.720605 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.720569    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5gv7\" (UniqueName: \"kubernetes.io/projected/053647d8-73f9-462d-97cd-cc771937306d-kube-api-access-b5gv7\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmhgn\" (UID: \"053647d8-73f9-462d-97cd-cc771937306d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:30.720605 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.720596    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/053647d8-73f9-462d-97cd-cc771937306d-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmhgn\" (UID: \"053647d8-73f9-462d-97cd-cc771937306d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:30.822145 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.822061    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/053647d8-73f9-462d-97cd-cc771937306d-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmhgn\" (UID: \"053647d8-73f9-462d-97cd-cc771937306d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:30.822145 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.822102    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b5gv7\" (UniqueName: \"kubernetes.io/projected/053647d8-73f9-462d-97cd-cc771937306d-kube-api-access-b5gv7\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmhgn\" (UID: \"053647d8-73f9-462d-97cd-cc771937306d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:30.822145 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.822120    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/053647d8-73f9-462d-97cd-cc771937306d-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmhgn\" (UID: \"053647d8-73f9-462d-97cd-cc771937306d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:30.822428 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.822159    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/053647d8-73f9-462d-97cd-cc771937306d-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmhgn\" (UID: \"053647d8-73f9-462d-97cd-cc771937306d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:30.822428 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.822209    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/053647d8-73f9-462d-97cd-cc771937306d-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmhgn\" (UID: \"053647d8-73f9-462d-97cd-cc771937306d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:30.822428 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.822238    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/053647d8-73f9-462d-97cd-cc771937306d-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmhgn\" (UID: \"053647d8-73f9-462d-97cd-cc771937306d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:30.822428 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:20:30.822263    2571 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found
May 25 14:20:30.822428 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:20:30.822374    2571 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found
May 25 14:20:30.822428 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:20:30.822392    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/053647d8-73f9-462d-97cd-cc771937306d-online-tls podName:053647d8-73f9-462d-97cd-cc771937306d nodeName:}" failed. No retries permitted until 2026-05-25 14:20:31.322368446 +0000 UTC m=+1239.157224533 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/053647d8-73f9-462d-97cd-cc771937306d-online-tls") pod "feast-simple-feast-remote-setup-5cff94746b-lmhgn" (UID: "053647d8-73f9-462d-97cd-cc771937306d") : secret "feast-simple-feast-remote-setup-online-tls" not found
May 25 14:20:30.822428 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:20:30.822419    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/053647d8-73f9-462d-97cd-cc771937306d-offline-tls podName:053647d8-73f9-462d-97cd-cc771937306d nodeName:}" failed. No retries permitted until 2026-05-25 14:20:31.322409129 +0000 UTC m=+1239.157265206 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/053647d8-73f9-462d-97cd-cc771937306d-offline-tls") pod "feast-simple-feast-remote-setup-5cff94746b-lmhgn" (UID: "053647d8-73f9-462d-97cd-cc771937306d") : secret "feast-simple-feast-remote-setup-offline-tls" not found
May 25 14:20:30.823039 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.823010    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/053647d8-73f9-462d-97cd-cc771937306d-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmhgn\" (UID: \"053647d8-73f9-462d-97cd-cc771937306d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:30.823153 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.823130    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/053647d8-73f9-462d-97cd-cc771937306d-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmhgn\" (UID: \"053647d8-73f9-462d-97cd-cc771937306d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:30.825003 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.824976    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/053647d8-73f9-462d-97cd-cc771937306d-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmhgn\" (UID: \"053647d8-73f9-462d-97cd-cc771937306d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:30.838508 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:30.838465    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5gv7\" (UniqueName: \"kubernetes.io/projected/053647d8-73f9-462d-97cd-cc771937306d-kube-api-access-b5gv7\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmhgn\" (UID: \"053647d8-73f9-462d-97cd-cc771937306d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:31.326721 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:31.326651    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/053647d8-73f9-462d-97cd-cc771937306d-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmhgn\" (UID: \"053647d8-73f9-462d-97cd-cc771937306d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:31.327048 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:31.326767    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/053647d8-73f9-462d-97cd-cc771937306d-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmhgn\" (UID: \"053647d8-73f9-462d-97cd-cc771937306d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:31.329972 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:31.329892    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/053647d8-73f9-462d-97cd-cc771937306d-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmhgn\" (UID: \"053647d8-73f9-462d-97cd-cc771937306d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:31.329972 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:31.329908    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/053647d8-73f9-462d-97cd-cc771937306d-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmhgn\" (UID: \"053647d8-73f9-462d-97cd-cc771937306d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:31.583920 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:31.583817    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:31.736169 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:31.736140    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"]
May 25 14:20:31.738399 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:20:31.738362    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod053647d8_73f9_462d_97cd_cc771937306d.slice/crio-62f17ca6947b9f72bb02d8ee70c269be37ba8557960960c4e1cf9b274f35868f WatchSource:0}: Error finding container 62f17ca6947b9f72bb02d8ee70c269be37ba8557960960c4e1cf9b274f35868f: Status 404 returned error can't find the container with id 62f17ca6947b9f72bb02d8ee70c269be37ba8557960960c4e1cf9b274f35868f
May 25 14:20:32.060689 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:32.060629    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" event={"ID":"053647d8-73f9-462d-97cd-cc771937306d","Type":"ContainerStarted","Data":"dfaca32e7afd7475d1c9138596110c051ecc9a16d580eeeeb20131faabe05acc"}
May 25 14:20:32.061023 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:32.060969    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" event={"ID":"053647d8-73f9-462d-97cd-cc771937306d","Type":"ContainerStarted","Data":"62f17ca6947b9f72bb02d8ee70c269be37ba8557960960c4e1cf9b274f35868f"}
May 25 14:20:37.078913 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:37.078872    2571 generic.go:358] "Generic (PLEG): container finished" podID="053647d8-73f9-462d-97cd-cc771937306d" containerID="dfaca32e7afd7475d1c9138596110c051ecc9a16d580eeeeb20131faabe05acc" exitCode=0
May 25 14:20:37.079379 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:37.078964    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" event={"ID":"053647d8-73f9-462d-97cd-cc771937306d","Type":"ContainerDied","Data":"dfaca32e7afd7475d1c9138596110c051ecc9a16d580eeeeb20131faabe05acc"}
May 25 14:20:38.084465 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:38.084421    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" event={"ID":"053647d8-73f9-462d-97cd-cc771937306d","Type":"ContainerStarted","Data":"357a84bbc3b69d5e58b49084f46c3c2ee54544d5be2674f6fbb25003fc906e7f"}
May 25 14:20:44.108662 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:44.108623    2571 generic.go:358] "Generic (PLEG): container finished" podID="053647d8-73f9-462d-97cd-cc771937306d" containerID="357a84bbc3b69d5e58b49084f46c3c2ee54544d5be2674f6fbb25003fc906e7f" exitCode=0
May 25 14:20:44.109084 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:44.108692    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" event={"ID":"053647d8-73f9-462d-97cd-cc771937306d","Type":"ContainerDied","Data":"357a84bbc3b69d5e58b49084f46c3c2ee54544d5be2674f6fbb25003fc906e7f"}
May 25 14:20:45.115369 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:45.115329    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" event={"ID":"053647d8-73f9-462d-97cd-cc771937306d","Type":"ContainerStarted","Data":"6ce2b6df332909a6f94b0bcad97c47ef2f9efef020e36b0a6939a3d13aa144d3"}
May 25 14:20:45.115888 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:45.115380    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" event={"ID":"053647d8-73f9-462d-97cd-cc771937306d","Type":"ContainerStarted","Data":"7297da02dc78224ed5791d779c341b099776f7b6bfbfdbb3403d495cd54ac3e1"}
May 25 14:20:45.115888 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:45.115391    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" event={"ID":"053647d8-73f9-462d-97cd-cc771937306d","Type":"ContainerStarted","Data":"eddc8e686a25620583d86ee9382ba096b53eec11f8175ff7f094883460896e88"}
May 25 14:20:45.144268 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:45.144197    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" podStartSLOduration=15.144177539 podStartE2EDuration="15.144177539s" podCreationTimestamp="2026-05-25 14:20:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 14:20:45.143004081 +0000 UTC m=+1252.977860174" watchObservedRunningTime="2026-05-25 14:20:45.144177539 +0000 UTC m=+1252.979033633"
May 25 14:20:46.584058 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:46.584003    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:46.584058 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:46.584063    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:46.584589 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:46.584077    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:46.585799 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:46.585757    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.36:8816: connect: connection refused"
May 25 14:20:46.585986 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:46.585803    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused"
May 25 14:20:46.585986 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:46.585799    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused"
May 25 14:20:49.585075 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:49.585022    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused"
May 25 14:20:49.585498 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:49.585022    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.36:8816: connect: connection refused"
May 25 14:20:49.585498 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:49.585034    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused"
May 25 14:20:51.584870 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:51.584820    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:51.584870 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:51.584876    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:51.584870 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:51.584886    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:52.585638 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:52.585601    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:52.586130 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:52.585871    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:52.586423 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:52.586402    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:52.586486 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:52.586469    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:52.590022 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:52.590000    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:20:53.145945 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:20:53.145890    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:21:56.135401 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:21:56.135373    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"]
May 25 14:21:56.135919 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:21:56.135657    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="online" containerID="cri-o://eddc8e686a25620583d86ee9382ba096b53eec11f8175ff7f094883460896e88" gracePeriod=30
May 25 14:21:56.135919 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:21:56.135707    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="ui" containerID="cri-o://6ce2b6df332909a6f94b0bcad97c47ef2f9efef020e36b0a6939a3d13aa144d3" gracePeriod=30
May 25 14:21:56.135919 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:21:56.135811    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="offline" containerID="cri-o://7297da02dc78224ed5791d779c341b099776f7b6bfbfdbb3403d495cd54ac3e1" gracePeriod=30
May 25 14:21:56.283353 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:21:56.283321    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"]
May 25 14:21:56.283730 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:21:56.283699    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="registry" containerID="cri-o://e19c49fa409d84064f36be2ff44eb8609caac683b0948bfae34b0f9c75e34d4a" gracePeriod=30
May 25 14:21:56.283964 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:21:56.283777    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="offline" containerID="cri-o://a58e415ff51ef4e9ebc494f97895d2df5997aea6d3bdd5139af6dbfe200241d0" gracePeriod=30
May 25 14:21:56.284075 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:21:56.283785    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="ui" containerID="cri-o://ac4e6b36648b38fbcd0464297b928ce01d50bb688519b22d3da752d7de6f0104" gracePeriod=30
May 25 14:21:56.284075 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:21:56.283845    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="online" containerID="cri-o://49ff8a6e57f5bc8aa65c37813ad0723cef60e221e5596d16f56dba7a761ad461" gracePeriod=30
May 25 14:21:57.163273 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:21:57.163189    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused"
May 25 14:21:57.342049 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:21:57.342016    2571 generic.go:358] "Generic (PLEG): container finished" podID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerID="ac4e6b36648b38fbcd0464297b928ce01d50bb688519b22d3da752d7de6f0104" exitCode=0
May 25 14:21:57.342049 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:21:57.342045    2571 generic.go:358] "Generic (PLEG): container finished" podID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerID="49ff8a6e57f5bc8aa65c37813ad0723cef60e221e5596d16f56dba7a761ad461" exitCode=0
May 25 14:21:57.342253 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:21:57.342080    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" event={"ID":"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad","Type":"ContainerDied","Data":"ac4e6b36648b38fbcd0464297b928ce01d50bb688519b22d3da752d7de6f0104"}
May 25 14:21:57.342253 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:21:57.342115    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" event={"ID":"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad","Type":"ContainerDied","Data":"49ff8a6e57f5bc8aa65c37813ad0723cef60e221e5596d16f56dba7a761ad461"}
May 25 14:21:57.344597 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:21:57.344575    2571 generic.go:358] "Generic (PLEG): container finished" podID="053647d8-73f9-462d-97cd-cc771937306d" containerID="6ce2b6df332909a6f94b0bcad97c47ef2f9efef020e36b0a6939a3d13aa144d3" exitCode=0
May 25 14:21:57.344597 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:21:57.344595    2571 generic.go:358] "Generic (PLEG): container finished" podID="053647d8-73f9-462d-97cd-cc771937306d" containerID="eddc8e686a25620583d86ee9382ba096b53eec11f8175ff7f094883460896e88" exitCode=0
May 25 14:21:57.344735 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:21:57.344647    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" event={"ID":"053647d8-73f9-462d-97cd-cc771937306d","Type":"ContainerDied","Data":"6ce2b6df332909a6f94b0bcad97c47ef2f9efef020e36b0a6939a3d13aa144d3"}
May 25 14:21:57.344735 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:21:57.344672    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" event={"ID":"053647d8-73f9-462d-97cd-cc771937306d","Type":"ContainerDied","Data":"eddc8e686a25620583d86ee9382ba096b53eec11f8175ff7f094883460896e88"}
May 25 14:21:57.829796 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:21:57.829759    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
May 25 14:22:02.587093 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:02.587007    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused"
May 25 14:22:03.143683 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:03.143643    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused"
May 25 14:22:07.163264 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:07.163209    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused"
May 25 14:22:07.830420 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:07.830379    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
May 25 14:22:12.586886 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:12.586841    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused"
May 25 14:22:13.143683 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:13.143643    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused"
May 25 14:22:17.162507 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:17.162462    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused"
May 25 14:22:17.162926 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:17.162582    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:22:17.829651 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:17.829606    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
May 25 14:22:17.829838 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:17.829732    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:22:22.586853 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:22.586811    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused"
May 25 14:22:22.587297 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:22.586992    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:22:23.143897 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:23.143851    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused"
May 25 14:22:23.144093 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:23.143990    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:22:26.439547 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:26.439519    2571 generic.go:358] "Generic (PLEG): container finished" podID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerID="a58e415ff51ef4e9ebc494f97895d2df5997aea6d3bdd5139af6dbfe200241d0" exitCode=137
May 25 14:22:26.439547 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:26.439546    2571 generic.go:358] "Generic (PLEG): container finished" podID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerID="e19c49fa409d84064f36be2ff44eb8609caac683b0948bfae34b0f9c75e34d4a" exitCode=137
May 25 14:22:26.439986 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:26.439594    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" event={"ID":"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad","Type":"ContainerDied","Data":"a58e415ff51ef4e9ebc494f97895d2df5997aea6d3bdd5139af6dbfe200241d0"}
May 25 14:22:26.439986 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:26.439628    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" event={"ID":"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad","Type":"ContainerDied","Data":"e19c49fa409d84064f36be2ff44eb8609caac683b0948bfae34b0f9c75e34d4a"}
May 25 14:22:26.442045 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:26.442025    2571 generic.go:358] "Generic (PLEG): container finished" podID="053647d8-73f9-462d-97cd-cc771937306d" containerID="7297da02dc78224ed5791d779c341b099776f7b6bfbfdbb3403d495cd54ac3e1" exitCode=137
May 25 14:22:26.442152 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:26.442051    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" event={"ID":"053647d8-73f9-462d-97cd-cc771937306d","Type":"ContainerDied","Data":"7297da02dc78224ed5791d779c341b099776f7b6bfbfdbb3403d495cd54ac3e1"}
May 25 14:22:26.788606 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:26.788579    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:22:26.927316 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:26.927294    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:22:26.938247 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:26.938228    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/053647d8-73f9-462d-97cd-cc771937306d-offline-tls\") pod \"053647d8-73f9-462d-97cd-cc771937306d\" (UID: \"053647d8-73f9-462d-97cd-cc771937306d\") "
May 25 14:22:26.938340 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:26.938255    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/053647d8-73f9-462d-97cd-cc771937306d-online-tls\") pod \"053647d8-73f9-462d-97cd-cc771937306d\" (UID: \"053647d8-73f9-462d-97cd-cc771937306d\") "
May 25 14:22:26.938340 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:26.938282    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5gv7\" (UniqueName: \"kubernetes.io/projected/053647d8-73f9-462d-97cd-cc771937306d-kube-api-access-b5gv7\") pod \"053647d8-73f9-462d-97cd-cc771937306d\" (UID: \"053647d8-73f9-462d-97cd-cc771937306d\") "
May 25 14:22:26.938340 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:26.938310    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/053647d8-73f9-462d-97cd-cc771937306d-registry-tls\") pod \"053647d8-73f9-462d-97cd-cc771937306d\" (UID: \"053647d8-73f9-462d-97cd-cc771937306d\") "
May 25 14:22:26.938477 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:26.938357    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/053647d8-73f9-462d-97cd-cc771937306d-feast-data\") pod \"053647d8-73f9-462d-97cd-cc771937306d\" (UID: \"053647d8-73f9-462d-97cd-cc771937306d\") "
May 25 14:22:26.938477 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:26.938385    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/053647d8-73f9-462d-97cd-cc771937306d-ui-tls\") pod \"053647d8-73f9-462d-97cd-cc771937306d\" (UID: \"053647d8-73f9-462d-97cd-cc771937306d\") "
May 25 14:22:26.938695 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:26.938663    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/053647d8-73f9-462d-97cd-cc771937306d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "053647d8-73f9-462d-97cd-cc771937306d" (UID: "053647d8-73f9-462d-97cd-cc771937306d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:22:26.938911 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:26.938838    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/053647d8-73f9-462d-97cd-cc771937306d-feast-data" (OuterVolumeSpecName: "feast-data") pod "053647d8-73f9-462d-97cd-cc771937306d" (UID: "053647d8-73f9-462d-97cd-cc771937306d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 14:22:26.940567 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:26.940539    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/053647d8-73f9-462d-97cd-cc771937306d-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "053647d8-73f9-462d-97cd-cc771937306d" (UID: "053647d8-73f9-462d-97cd-cc771937306d"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:22:26.940659 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:26.940568    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/053647d8-73f9-462d-97cd-cc771937306d-online-tls" (OuterVolumeSpecName: "online-tls") pod "053647d8-73f9-462d-97cd-cc771937306d" (UID: "053647d8-73f9-462d-97cd-cc771937306d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:22:26.940659 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:26.940599    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/053647d8-73f9-462d-97cd-cc771937306d-kube-api-access-b5gv7" (OuterVolumeSpecName: "kube-api-access-b5gv7") pod "053647d8-73f9-462d-97cd-cc771937306d" (UID: "053647d8-73f9-462d-97cd-cc771937306d"). InnerVolumeSpecName "kube-api-access-b5gv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 14:22:26.940731 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:26.940693    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/053647d8-73f9-462d-97cd-cc771937306d-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "053647d8-73f9-462d-97cd-cc771937306d" (UID: "053647d8-73f9-462d-97cd-cc771937306d"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:22:27.039771 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.039697    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-offline-tls\") pod \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\" (UID: \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\") "
May 25 14:22:27.039771 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.039744    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-475dc\" (UniqueName: \"kubernetes.io/projected/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-kube-api-access-475dc\") pod \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\" (UID: \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\") "
May 25 14:22:27.039771 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.039768    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-registry-tls\") pod \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\" (UID: \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\") "
May 25 14:22:27.040039 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.039870    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-ui-tls\") pod \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\" (UID: \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\") "
May 25 14:22:27.040039 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.039927    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-feast-data\") pod \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\" (UID: \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\") "
May 25 14:22:27.040039 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.039992    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-online-tls\") pod \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\" (UID: \"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad\") "
May 25 14:22:27.040288 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.040266    2571 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/053647d8-73f9-462d-97cd-cc771937306d-offline-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:22:27.040374 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.040295    2571 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/053647d8-73f9-462d-97cd-cc771937306d-online-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:22:27.040374 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.040308    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5gv7\" (UniqueName: \"kubernetes.io/projected/053647d8-73f9-462d-97cd-cc771937306d-kube-api-access-b5gv7\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:22:27.040374 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.040323    2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/053647d8-73f9-462d-97cd-cc771937306d-registry-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:22:27.040374 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.040336    2571 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/053647d8-73f9-462d-97cd-cc771937306d-feast-data\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:22:27.040374 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.040349    2571 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/053647d8-73f9-462d-97cd-cc771937306d-ui-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:22:27.040633 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.040511    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-feast-data" (OuterVolumeSpecName: "feast-data") pod "87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" (UID: "87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 14:22:27.042046 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.042017    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-kube-api-access-475dc" (OuterVolumeSpecName: "kube-api-access-475dc") pod "87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" (UID: "87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad"). InnerVolumeSpecName "kube-api-access-475dc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 14:22:27.042362 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.042341    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" (UID: "87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:22:27.042599 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.042579    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" (UID: "87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:22:27.042669 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.042594    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" (UID: "87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:22:27.042669 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.042608    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-online-tls" (OuterVolumeSpecName: "online-tls") pod "87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" (UID: "87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:22:27.141424 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.141382    2571 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-feast-data\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:22:27.141424 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.141421    2571 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-online-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:22:27.141424 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.141434    2571 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-offline-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:22:27.141633 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.141447    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-475dc\" (UniqueName: \"kubernetes.io/projected/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-kube-api-access-475dc\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:22:27.141633 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.141461    2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-registry-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:22:27.141633 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.141472    2571 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad-ui-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:22:27.447764 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.447738    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"
May 25 14:22:27.448236 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.447735    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm" event={"ID":"87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad","Type":"ContainerDied","Data":"847380af4ee1823bf4b39b9dbcd4c9715d0f2881d6c9969b03922f8b4159d23d"}
May 25 14:22:27.448236 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.447867    2571 scope.go:117] "RemoveContainer" containerID="ac4e6b36648b38fbcd0464297b928ce01d50bb688519b22d3da752d7de6f0104"
May 25 14:22:27.450360 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.450323    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn" event={"ID":"053647d8-73f9-462d-97cd-cc771937306d","Type":"ContainerDied","Data":"62f17ca6947b9f72bb02d8ee70c269be37ba8557960960c4e1cf9b274f35868f"}
May 25 14:22:27.450360 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.450347    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"
May 25 14:22:27.457623 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.457607    2571 scope.go:117] "RemoveContainer" containerID="a58e415ff51ef4e9ebc494f97895d2df5997aea6d3bdd5139af6dbfe200241d0"
May 25 14:22:27.464822 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.464805    2571 scope.go:117] "RemoveContainer" containerID="49ff8a6e57f5bc8aa65c37813ad0723cef60e221e5596d16f56dba7a761ad461"
May 25 14:22:27.471409 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.471384    2571 scope.go:117] "RemoveContainer" containerID="e19c49fa409d84064f36be2ff44eb8609caac683b0948bfae34b0f9c75e34d4a"
May 25 14:22:27.474226 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.474205    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"]
May 25 14:22:27.476864 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.476838    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-9q5jm"]
May 25 14:22:27.479574 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.479558    2571 scope.go:117] "RemoveContainer" containerID="aa56a6cc877dd6da02d56663347d9f23651542367184f55efb460f868c1be6b9"
May 25 14:22:27.486338 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.486321    2571 scope.go:117] "RemoveContainer" containerID="ddeea4e535e5eec2b189dc8e26c65dfd28a53f4b4a2ac8892b88160ca21ec209"
May 25 14:22:27.488583 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.488562    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"]
May 25 14:22:27.493534 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.493516    2571 scope.go:117] "RemoveContainer" containerID="6ce2b6df332909a6f94b0bcad97c47ef2f9efef020e36b0a6939a3d13aa144d3"
May 25 14:22:27.495033 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.494519    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmhgn"]
May 25 14:22:27.500771 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.500753    2571 scope.go:117] "RemoveContainer" containerID="7297da02dc78224ed5791d779c341b099776f7b6bfbfdbb3403d495cd54ac3e1"
May 25 14:22:27.507296 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.507280    2571 scope.go:117] "RemoveContainer" containerID="eddc8e686a25620583d86ee9382ba096b53eec11f8175ff7f094883460896e88"
May 25 14:22:27.513683 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.513668    2571 scope.go:117] "RemoveContainer" containerID="357a84bbc3b69d5e58b49084f46c3c2ee54544d5be2674f6fbb25003fc906e7f"
May 25 14:22:27.520287 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:27.520262    2571 scope.go:117] "RemoveContainer" containerID="dfaca32e7afd7475d1c9138596110c051ecc9a16d580eeeeb20131faabe05acc"
May 25 14:22:28.825984 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:28.825925    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="053647d8-73f9-462d-97cd-cc771937306d" path="/var/lib/kubelet/pods/053647d8-73f9-462d-97cd-cc771937306d/volumes"
May 25 14:22:28.826525 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:22:28.826509    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" path="/var/lib/kubelet/pods/87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad/volumes"
May 25 14:23:08.054538 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.054501    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"]
May 25 14:23:08.055042 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.054829    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="feast-apply"
May 25 14:23:08.055042 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.054839    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="feast-apply"
May 25 14:23:08.055042 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.054846    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="online"
May 25 14:23:08.055042 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.054851    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="online"
May 25 14:23:08.055042 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.054863    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="feast-init"
May 25 14:23:08.055042 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.054869    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="feast-init"
May 25 14:23:08.055042 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.054878    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="offline"
May 25 14:23:08.055042 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.054883    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="offline"
May 25 14:23:08.055042 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.054892    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="ui"
May 25 14:23:08.055042 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.054898    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="ui"
May 25 14:23:08.055042 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.054903    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="registry"
May 25 14:23:08.055042 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.054909    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="registry"
May 25 14:23:08.055042 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.054916    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="offline"
May 25 14:23:08.055042 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.054922    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="offline"
May 25 14:23:08.055042 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.054950    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="ui"
May 25 14:23:08.055042 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.054960    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="ui"
May 25 14:23:08.055042 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.054977    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="feast-init"
May 25 14:23:08.055042 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.054985    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="feast-init"
May 25 14:23:08.055042 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.054995    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="online"
May 25 14:23:08.055042 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.055007    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="online"
May 25 14:23:08.055042 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.055016    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="feast-apply"
May 25 14:23:08.055042 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.055033    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="feast-apply"
May 25 14:23:08.055944 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.055112    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="online"
May 25 14:23:08.055944 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.055122    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="online"
May 25 14:23:08.055944 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.055130    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="offline"
May 25 14:23:08.055944 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.055136    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="registry"
May 25 14:23:08.055944 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.055143    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="ui"
May 25 14:23:08.055944 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.055153    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="053647d8-73f9-462d-97cd-cc771937306d" containerName="offline"
May 25 14:23:08.055944 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.055164    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="87cfdfae-aa95-4b1d-8f0d-f9c76021b9ad" containerName="ui"
May 25 14:23:08.060208 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.060190    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:08.062481 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.062458    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 25 14:23:08.062481 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.062472    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 25 14:23:08.062673 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.062662    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-tkwrt\""
May 25 14:23:08.062899 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.062880    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 25 14:23:08.063124 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.063107    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 25 14:23:08.063177 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.063147    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 25 14:23:08.063391 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.063353    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 25 14:23:08.073267 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.073244    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"]
May 25 14:23:08.164157 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.164119    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-845xs\" (UniqueName: \"kubernetes.io/projected/28e27d47-f841-43ad-a928-814ca3435c37-kube-api-access-845xs\") pod \"feast-simple-feast-setup-664fcfcc58-zzkbc\" (UID: \"28e27d47-f841-43ad-a928-814ca3435c37\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:08.164325 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.164166    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/28e27d47-f841-43ad-a928-814ca3435c37-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-zzkbc\" (UID: \"28e27d47-f841-43ad-a928-814ca3435c37\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:08.164325 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.164232    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zzkbc\" (UID: \"28e27d47-f841-43ad-a928-814ca3435c37\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:08.164325 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.164305    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zzkbc\" (UID: \"28e27d47-f841-43ad-a928-814ca3435c37\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:08.164458 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.164345    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zzkbc\" (UID: \"28e27d47-f841-43ad-a928-814ca3435c37\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:08.164458 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.164366    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zzkbc\" (UID: \"28e27d47-f841-43ad-a928-814ca3435c37\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:08.265181 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.265154    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zzkbc\" (UID: \"28e27d47-f841-43ad-a928-814ca3435c37\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:08.265181 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.265181    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zzkbc\" (UID: \"28e27d47-f841-43ad-a928-814ca3435c37\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:08.265381 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.265213    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-845xs\" (UniqueName: \"kubernetes.io/projected/28e27d47-f841-43ad-a928-814ca3435c37-kube-api-access-845xs\") pod \"feast-simple-feast-setup-664fcfcc58-zzkbc\" (UID: \"28e27d47-f841-43ad-a928-814ca3435c37\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:08.265381 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.265247    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/28e27d47-f841-43ad-a928-814ca3435c37-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-zzkbc\" (UID: \"28e27d47-f841-43ad-a928-814ca3435c37\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:08.265381 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.265285    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zzkbc\" (UID: \"28e27d47-f841-43ad-a928-814ca3435c37\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:08.265381 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.265316    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zzkbc\" (UID: \"28e27d47-f841-43ad-a928-814ca3435c37\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:08.265584 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:23:08.265416    2571 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found
May 25 14:23:08.265584 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:23:08.265518    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-online-tls podName:28e27d47-f841-43ad-a928-814ca3435c37 nodeName:}" failed. No retries permitted until 2026-05-25 14:23:08.765487278 +0000 UTC m=+1396.600343351 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-online-tls") pod "feast-simple-feast-setup-664fcfcc58-zzkbc" (UID: "28e27d47-f841-43ad-a928-814ca3435c37") : secret "feast-simple-feast-setup-online-tls" not found
May 25 14:23:08.265706 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.265656    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/28e27d47-f841-43ad-a928-814ca3435c37-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-zzkbc\" (UID: \"28e27d47-f841-43ad-a928-814ca3435c37\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:08.267953 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.267897    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zzkbc\" (UID: \"28e27d47-f841-43ad-a928-814ca3435c37\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:08.267953 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.267926    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zzkbc\" (UID: \"28e27d47-f841-43ad-a928-814ca3435c37\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:08.268098 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.267970    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zzkbc\" (UID: \"28e27d47-f841-43ad-a928-814ca3435c37\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:08.273880 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.273862    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-845xs\" (UniqueName: \"kubernetes.io/projected/28e27d47-f841-43ad-a928-814ca3435c37-kube-api-access-845xs\") pod \"feast-simple-feast-setup-664fcfcc58-zzkbc\" (UID: \"28e27d47-f841-43ad-a928-814ca3435c37\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:08.768670 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.768638    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zzkbc\" (UID: \"28e27d47-f841-43ad-a928-814ca3435c37\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:08.771228 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.771197    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zzkbc\" (UID: \"28e27d47-f841-43ad-a928-814ca3435c37\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:08.982044 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:08.982021    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:09.115768 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:09.115745    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"]
May 25 14:23:09.117867 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:23:09.117840    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod28e27d47_f841_43ad_a928_814ca3435c37.slice/crio-88092cb8365454ab73a16af11de5dc0f9d0f9b0c7eab17a45cc7e581088eae7d WatchSource:0}: Error finding container 88092cb8365454ab73a16af11de5dc0f9d0f9b0c7eab17a45cc7e581088eae7d: Status 404 returned error can't find the container with id 88092cb8365454ab73a16af11de5dc0f9d0f9b0c7eab17a45cc7e581088eae7d
May 25 14:23:09.119493 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:09.119480    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 25 14:23:09.579263 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:09.579227    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" event={"ID":"28e27d47-f841-43ad-a928-814ca3435c37","Type":"ContainerStarted","Data":"813eea535a3542cd57f18d23d11d7c8a7c83a51c3b4b42e590cb08a7eeb969d1"}
May 25 14:23:09.579432 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:09.579270    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" event={"ID":"28e27d47-f841-43ad-a928-814ca3435c37","Type":"ContainerStarted","Data":"88092cb8365454ab73a16af11de5dc0f9d0f9b0c7eab17a45cc7e581088eae7d"}
May 25 14:23:14.596452 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:14.596415    2571 generic.go:358] "Generic (PLEG): container finished" podID="28e27d47-f841-43ad-a928-814ca3435c37" containerID="813eea535a3542cd57f18d23d11d7c8a7c83a51c3b4b42e590cb08a7eeb969d1" exitCode=0
May 25 14:23:14.596833 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:14.596489    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" event={"ID":"28e27d47-f841-43ad-a928-814ca3435c37","Type":"ContainerDied","Data":"813eea535a3542cd57f18d23d11d7c8a7c83a51c3b4b42e590cb08a7eeb969d1"}
May 25 14:23:15.602243 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:15.602211    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" event={"ID":"28e27d47-f841-43ad-a928-814ca3435c37","Type":"ContainerStarted","Data":"4affff872e8e34f001f75bfcdd229187395f80afed3416ca438c6cb88eeffc5a"}
May 25 14:23:20.623473 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:20.623442    2571 generic.go:358] "Generic (PLEG): container finished" podID="28e27d47-f841-43ad-a928-814ca3435c37" containerID="4affff872e8e34f001f75bfcdd229187395f80afed3416ca438c6cb88eeffc5a" exitCode=0
May 25 14:23:20.623923 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:20.623527    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" event={"ID":"28e27d47-f841-43ad-a928-814ca3435c37","Type":"ContainerDied","Data":"4affff872e8e34f001f75bfcdd229187395f80afed3416ca438c6cb88eeffc5a"}
May 25 14:23:21.636601 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:21.636554    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" event={"ID":"28e27d47-f841-43ad-a928-814ca3435c37","Type":"ContainerStarted","Data":"8894567ac68e20c6789c1d2d352339c44b32e2f0247c8be293aeecdb80d26948"}
May 25 14:23:21.636601 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:21.636597    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" event={"ID":"28e27d47-f841-43ad-a928-814ca3435c37","Type":"ContainerStarted","Data":"a9aa842a386362d6b7f96d21a323bd5ed03f968f6f8f4423a95cc2d79fc29a95"}
May 25 14:23:21.637158 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:21.636614    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" event={"ID":"28e27d47-f841-43ad-a928-814ca3435c37","Type":"ContainerStarted","Data":"b4e02781b5b7d4e909591c0e40648d0352a7181f24a5240576af02fc695e29c2"}
May 25 14:23:21.637158 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:21.636628    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" event={"ID":"28e27d47-f841-43ad-a928-814ca3435c37","Type":"ContainerStarted","Data":"5be60284dcb878e0c63e3262ed62d1ac6afd36e9131260045ad19694c3c0bd1e"}
May 25 14:23:21.663185 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:21.663095    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" podStartSLOduration=13.663074372 podStartE2EDuration="13.663074372s" podCreationTimestamp="2026-05-25 14:23:08 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 14:23:21.660005412 +0000 UTC m=+1409.494861504" watchObservedRunningTime="2026-05-25 14:23:21.663074372 +0000 UTC m=+1409.497930466"
May 25 14:23:23.982621 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:23.982478    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:23.982621 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:23.982522    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:23.982621 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:23.982536    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:23.982621 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:23.982549    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:23.984956 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:23.984748    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.37:6571: connect: connection refused"
May 25 14:23:23.984956 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:23.984782    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused"
May 25 14:23:23.984956 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:23.984800    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused"
May 25 14:23:23.984956 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:23.984859    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.37:8816: connect: connection refused"
May 25 14:23:26.983209 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:26.983181    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:26.983641 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:26.983269    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:26.983641 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:26.983571    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:26.984092 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:26.984068    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:26.984230 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:26.984098    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:26.984230 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:26.984114    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:26.984420 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:26.984400    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:26.984698 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:26.984681    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:26.984766 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:26.984737    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:26.988316 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:26.988295    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:27.656563 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:27.656536    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:23:27.659442 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:23:27.659423    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:24:28.735403 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:28.735371    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"]
May 25 14:24:28.735980 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:28.735793    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="registry" containerID="cri-o://5be60284dcb878e0c63e3262ed62d1ac6afd36e9131260045ad19694c3c0bd1e" gracePeriod=30
May 25 14:24:28.735980 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:28.735875    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="ui" containerID="cri-o://8894567ac68e20c6789c1d2d352339c44b32e2f0247c8be293aeecdb80d26948" gracePeriod=30
May 25 14:24:28.735980 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:28.735964    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="offline" containerID="cri-o://a9aa842a386362d6b7f96d21a323bd5ed03f968f6f8f4423a95cc2d79fc29a95" gracePeriod=30
May 25 14:24:28.736134 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:28.736033    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="online" containerID="cri-o://b4e02781b5b7d4e909591c0e40648d0352a7181f24a5240576af02fc695e29c2" gracePeriod=30
May 25 14:24:29.185496 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.185459    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"]
May 25 14:24:29.189221 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.189199    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:29.191357 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.191339    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-9sjkg\""
May 25 14:24:29.204432 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.204405    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"]
May 25 14:24:29.317504 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.317468    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-hspxb\" (UID: \"e62d45b9-1363-415f-be2a-8cec44d72278\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:29.317663 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.317514    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgt66\" (UniqueName: \"kubernetes.io/projected/e62d45b9-1363-415f-be2a-8cec44d72278-kube-api-access-wgt66\") pod \"feast-simple-feast-setup-664fcfcc58-hspxb\" (UID: \"e62d45b9-1363-415f-be2a-8cec44d72278\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:29.317663 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.317607    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-hspxb\" (UID: \"e62d45b9-1363-415f-be2a-8cec44d72278\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:29.317663 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.317635    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-hspxb\" (UID: \"e62d45b9-1363-415f-be2a-8cec44d72278\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:29.317774 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.317676    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-hspxb\" (UID: \"e62d45b9-1363-415f-be2a-8cec44d72278\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:29.317774 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.317743    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e62d45b9-1363-415f-be2a-8cec44d72278-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-hspxb\" (UID: \"e62d45b9-1363-415f-be2a-8cec44d72278\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:29.418567 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.418504    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-hspxb\" (UID: \"e62d45b9-1363-415f-be2a-8cec44d72278\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:29.418567 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.418555    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-hspxb\" (UID: \"e62d45b9-1363-415f-be2a-8cec44d72278\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:29.418755 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.418575    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e62d45b9-1363-415f-be2a-8cec44d72278-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-hspxb\" (UID: \"e62d45b9-1363-415f-be2a-8cec44d72278\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:29.418755 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.418616    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-hspxb\" (UID: \"e62d45b9-1363-415f-be2a-8cec44d72278\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:29.418755 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.418639    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wgt66\" (UniqueName: \"kubernetes.io/projected/e62d45b9-1363-415f-be2a-8cec44d72278-kube-api-access-wgt66\") pod \"feast-simple-feast-setup-664fcfcc58-hspxb\" (UID: \"e62d45b9-1363-415f-be2a-8cec44d72278\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:29.418755 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:24:29.418646    2571 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found
May 25 14:24:29.418755 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.418694    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-hspxb\" (UID: \"e62d45b9-1363-415f-be2a-8cec44d72278\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:29.418755 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:24:29.418722    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-offline-tls podName:e62d45b9-1363-415f-be2a-8cec44d72278 nodeName:}" failed. No retries permitted until 2026-05-25 14:24:29.918705223 +0000 UTC m=+1477.753561295 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-offline-tls") pod "feast-simple-feast-setup-664fcfcc58-hspxb" (UID: "e62d45b9-1363-415f-be2a-8cec44d72278") : secret "feast-simple-feast-setup-offline-tls" not found
May 25 14:24:29.418755 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:24:29.418738    2571 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found
May 25 14:24:29.419134 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:24:29.418792    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-online-tls podName:e62d45b9-1363-415f-be2a-8cec44d72278 nodeName:}" failed. No retries permitted until 2026-05-25 14:24:29.918776004 +0000 UTC m=+1477.753632089 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-online-tls") pod "feast-simple-feast-setup-664fcfcc58-hspxb" (UID: "e62d45b9-1363-415f-be2a-8cec44d72278") : secret "feast-simple-feast-setup-online-tls" not found
May 25 14:24:29.419134 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.419038    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e62d45b9-1363-415f-be2a-8cec44d72278-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-hspxb\" (UID: \"e62d45b9-1363-415f-be2a-8cec44d72278\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:29.421255 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.421237    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-hspxb\" (UID: \"e62d45b9-1363-415f-be2a-8cec44d72278\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:29.421333 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.421301    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-hspxb\" (UID: \"e62d45b9-1363-415f-be2a-8cec44d72278\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:29.429727 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.429709    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgt66\" (UniqueName: \"kubernetes.io/projected/e62d45b9-1363-415f-be2a-8cec44d72278-kube-api-access-wgt66\") pod \"feast-simple-feast-setup-664fcfcc58-hspxb\" (UID: \"e62d45b9-1363-415f-be2a-8cec44d72278\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:29.842982 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.842951    2571 generic.go:358] "Generic (PLEG): container finished" podID="28e27d47-f841-43ad-a928-814ca3435c37" containerID="8894567ac68e20c6789c1d2d352339c44b32e2f0247c8be293aeecdb80d26948" exitCode=0
May 25 14:24:29.842982 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.842976    2571 generic.go:358] "Generic (PLEG): container finished" podID="28e27d47-f841-43ad-a928-814ca3435c37" containerID="b4e02781b5b7d4e909591c0e40648d0352a7181f24a5240576af02fc695e29c2" exitCode=0
May 25 14:24:29.843372 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.843001    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" event={"ID":"28e27d47-f841-43ad-a928-814ca3435c37","Type":"ContainerDied","Data":"8894567ac68e20c6789c1d2d352339c44b32e2f0247c8be293aeecdb80d26948"}
May 25 14:24:29.843372 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.843022    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" event={"ID":"28e27d47-f841-43ad-a928-814ca3435c37","Type":"ContainerDied","Data":"b4e02781b5b7d4e909591c0e40648d0352a7181f24a5240576af02fc695e29c2"}
May 25 14:24:29.922101 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.922070    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-hspxb\" (UID: \"e62d45b9-1363-415f-be2a-8cec44d72278\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:29.922238 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.922128    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-hspxb\" (UID: \"e62d45b9-1363-415f-be2a-8cec44d72278\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:29.924520 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.924495    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-hspxb\" (UID: \"e62d45b9-1363-415f-be2a-8cec44d72278\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:29.924621 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:29.924526    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-hspxb\" (UID: \"e62d45b9-1363-415f-be2a-8cec44d72278\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:30.100362 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:30.100283    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:30.232132 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:30.232103    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"]
May 25 14:24:30.235009 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:24:30.234983    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode62d45b9_1363_415f_be2a_8cec44d72278.slice/crio-bc03ceed7141b5a45b9e814d2341b145e802489eb9e165ea9fd6bb6457f451ce WatchSource:0}: Error finding container bc03ceed7141b5a45b9e814d2341b145e802489eb9e165ea9fd6bb6457f451ce: Status 404 returned error can't find the container with id bc03ceed7141b5a45b9e814d2341b145e802489eb9e165ea9fd6bb6457f451ce
May 25 14:24:30.846833 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:30.846795    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" event={"ID":"e62d45b9-1363-415f-be2a-8cec44d72278","Type":"ContainerStarted","Data":"86f83e18ea9cc4d89ed3130a61186c42c90652543a5b70e7b4ed4528c31a9f06"}
May 25 14:24:30.846833 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:30.846837    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" event={"ID":"e62d45b9-1363-415f-be2a-8cec44d72278","Type":"ContainerStarted","Data":"bc03ceed7141b5a45b9e814d2341b145e802489eb9e165ea9fd6bb6457f451ce"}
May 25 14:24:34.857712 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:34.857681    2571 generic.go:358] "Generic (PLEG): container finished" podID="e62d45b9-1363-415f-be2a-8cec44d72278" containerID="86f83e18ea9cc4d89ed3130a61186c42c90652543a5b70e7b4ed4528c31a9f06" exitCode=0
May 25 14:24:34.858047 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:34.857742    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" event={"ID":"e62d45b9-1363-415f-be2a-8cec44d72278","Type":"ContainerDied","Data":"86f83e18ea9cc4d89ed3130a61186c42c90652543a5b70e7b4ed4528c31a9f06"}
May 25 14:24:35.865070 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:35.865037    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" event={"ID":"e62d45b9-1363-415f-be2a-8cec44d72278","Type":"ContainerStarted","Data":"d02f710874e38dac58faa4857320c8f6d0e87c751fc9d09399e924f223769fcc"}
May 25 14:24:36.984203 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:36.984154    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused"
May 25 14:24:37.657622 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:37.657579    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused"
May 25 14:24:40.881156 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:40.881124    2571 generic.go:358] "Generic (PLEG): container finished" podID="e62d45b9-1363-415f-be2a-8cec44d72278" containerID="d02f710874e38dac58faa4857320c8f6d0e87c751fc9d09399e924f223769fcc" exitCode=0
May 25 14:24:40.881522 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:40.881193    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" event={"ID":"e62d45b9-1363-415f-be2a-8cec44d72278","Type":"ContainerDied","Data":"d02f710874e38dac58faa4857320c8f6d0e87c751fc9d09399e924f223769fcc"}
May 25 14:24:41.889478 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:41.889434    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" event={"ID":"e62d45b9-1363-415f-be2a-8cec44d72278","Type":"ContainerStarted","Data":"0887819ee016ef1655bf91444dfb19ff6a0f715af969e34d62db98707de50b6d"}
May 25 14:24:41.890002 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:41.889490    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" event={"ID":"e62d45b9-1363-415f-be2a-8cec44d72278","Type":"ContainerStarted","Data":"21e7899f14412f683669a52882af11e5615dbadfca66875e396c71752e0b64d2"}
May 25 14:24:41.890002 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:41.889506    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" event={"ID":"e62d45b9-1363-415f-be2a-8cec44d72278","Type":"ContainerStarted","Data":"936ae754df4725e4959388ede119b921301d2d31747f235ec5fb830ca6579f6a"}
May 25 14:24:41.890002 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:41.889520    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" event={"ID":"e62d45b9-1363-415f-be2a-8cec44d72278","Type":"ContainerStarted","Data":"1947ba2c9b3ebcd1b179a06a7a12a5a7c60075e6ba1801196a016e3ef124b3c0"}
May 25 14:24:41.913160 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:41.913090    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" podStartSLOduration=12.913071976 podStartE2EDuration="12.913071976s" podCreationTimestamp="2026-05-25 14:24:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 14:24:41.911431779 +0000 UTC m=+1489.746287943" watchObservedRunningTime="2026-05-25 14:24:41.913071976 +0000 UTC m=+1489.747928071"
May 25 14:24:42.101254 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:42.101218    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:42.101498 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:42.101467    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:42.101498 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:42.101494    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:42.101694 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:42.101507    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:42.103352 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:42.103314    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.38:8816: connect: connection refused"
May 25 14:24:42.103462 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:42.103417    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="online" probeResult="failure" output="Get \"https://10.134.0.38:6567/health\": dial tcp 10.134.0.38:6567: connect: connection refused"
May 25 14:24:42.103539 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:42.103515    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.38:8443: connect: connection refused"
May 25 14:24:42.103741 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:42.103539    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.38:6571: connect: connection refused"
May 25 14:24:45.101489 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:45.101430    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.38:8816: connect: connection refused"
May 25 14:24:45.102022 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:45.101431    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.38:8443: connect: connection refused"
May 25 14:24:45.102022 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:45.101434    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.38:6571: connect: connection refused"
May 25 14:24:45.102288 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:45.102248    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="online" probeResult="failure" output="Get \"https://10.134.0.38:6567/health\": dial tcp 10.134.0.38:6567: connect: connection refused"
May 25 14:24:46.984433 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:46.984379    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused"
May 25 14:24:47.657534 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:47.657496    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused"
May 25 14:24:48.101220 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:48.101188    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:48.107145 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:48.102015    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:48.107145 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:48.102505    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:48.107900 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:48.107818    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:48.107900 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:48.107861    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:48.107900 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:48.107877    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:48.108398 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:48.108375    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:48.108494 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:48.108446    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:48.108494 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:48.108487    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:48.110379 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:48.110360    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:48.912155 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:48.912125    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:48.914975 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:48.914958    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:24:56.984730 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:56.984685    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused"
May 25 14:24:56.985202 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:56.984827    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:24:57.657523 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:57.657473    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused"
May 25 14:24:57.657719 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:57.657613    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:24:58.943407 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:58.943370    2571 generic.go:358] "Generic (PLEG): container finished" podID="28e27d47-f841-43ad-a928-814ca3435c37" containerID="a9aa842a386362d6b7f96d21a323bd5ed03f968f6f8f4423a95cc2d79fc29a95" exitCode=137
May 25 14:24:58.943407 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:58.943400    2571 generic.go:358] "Generic (PLEG): container finished" podID="28e27d47-f841-43ad-a928-814ca3435c37" containerID="5be60284dcb878e0c63e3262ed62d1ac6afd36e9131260045ad19694c3c0bd1e" exitCode=137
May 25 14:24:58.943798 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:58.943437    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" event={"ID":"28e27d47-f841-43ad-a928-814ca3435c37","Type":"ContainerDied","Data":"a9aa842a386362d6b7f96d21a323bd5ed03f968f6f8f4423a95cc2d79fc29a95"}
May 25 14:24:58.943798 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:58.943470    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" event={"ID":"28e27d47-f841-43ad-a928-814ca3435c37","Type":"ContainerDied","Data":"5be60284dcb878e0c63e3262ed62d1ac6afd36e9131260045ad19694c3c0bd1e"}
May 25 14:24:59.379150 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.379126    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:24:59.477539 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.477512    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-ui-tls\") pod \"28e27d47-f841-43ad-a928-814ca3435c37\" (UID: \"28e27d47-f841-43ad-a928-814ca3435c37\") "
May 25 14:24:59.477539 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.477540    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-registry-tls\") pod \"28e27d47-f841-43ad-a928-814ca3435c37\" (UID: \"28e27d47-f841-43ad-a928-814ca3435c37\") "
May 25 14:24:59.477798 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.477568    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-offline-tls\") pod \"28e27d47-f841-43ad-a928-814ca3435c37\" (UID: \"28e27d47-f841-43ad-a928-814ca3435c37\") "
May 25 14:24:59.477798 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.477598    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/28e27d47-f841-43ad-a928-814ca3435c37-feast-data\") pod \"28e27d47-f841-43ad-a928-814ca3435c37\" (UID: \"28e27d47-f841-43ad-a928-814ca3435c37\") "
May 25 14:24:59.477798 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.477624    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-845xs\" (UniqueName: \"kubernetes.io/projected/28e27d47-f841-43ad-a928-814ca3435c37-kube-api-access-845xs\") pod \"28e27d47-f841-43ad-a928-814ca3435c37\" (UID: \"28e27d47-f841-43ad-a928-814ca3435c37\") "
May 25 14:24:59.477798 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.477639    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-online-tls\") pod \"28e27d47-f841-43ad-a928-814ca3435c37\" (UID: \"28e27d47-f841-43ad-a928-814ca3435c37\") "
May 25 14:24:59.478241 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.478195    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28e27d47-f841-43ad-a928-814ca3435c37-feast-data" (OuterVolumeSpecName: "feast-data") pod "28e27d47-f841-43ad-a928-814ca3435c37" (UID: "28e27d47-f841-43ad-a928-814ca3435c37"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 14:24:59.479984 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.479909    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "28e27d47-f841-43ad-a928-814ca3435c37" (UID: "28e27d47-f841-43ad-a928-814ca3435c37"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:24:59.480089 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.480021    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-online-tls" (OuterVolumeSpecName: "online-tls") pod "28e27d47-f841-43ad-a928-814ca3435c37" (UID: "28e27d47-f841-43ad-a928-814ca3435c37"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:24:59.480143 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.480121    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28e27d47-f841-43ad-a928-814ca3435c37-kube-api-access-845xs" (OuterVolumeSpecName: "kube-api-access-845xs") pod "28e27d47-f841-43ad-a928-814ca3435c37" (UID: "28e27d47-f841-43ad-a928-814ca3435c37"). InnerVolumeSpecName "kube-api-access-845xs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 14:24:59.480201 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.480170    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "28e27d47-f841-43ad-a928-814ca3435c37" (UID: "28e27d47-f841-43ad-a928-814ca3435c37"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:24:59.480254 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.480215    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "28e27d47-f841-43ad-a928-814ca3435c37" (UID: "28e27d47-f841-43ad-a928-814ca3435c37"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:24:59.578911 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.578853    2571 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-ui-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:24:59.578911 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.578876    2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-registry-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:24:59.578911 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.578886    2571 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-offline-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:24:59.578911 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.578895    2571 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/28e27d47-f841-43ad-a928-814ca3435c37-feast-data\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:24:59.578911 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.578903    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-845xs\" (UniqueName: \"kubernetes.io/projected/28e27d47-f841-43ad-a928-814ca3435c37-kube-api-access-845xs\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:24:59.578911 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.578912    2571 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/28e27d47-f841-43ad-a928-814ca3435c37-online-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:24:59.950316 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.950289    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"
May 25 14:24:59.950716 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.950292    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc" event={"ID":"28e27d47-f841-43ad-a928-814ca3435c37","Type":"ContainerDied","Data":"88092cb8365454ab73a16af11de5dc0f9d0f9b0c7eab17a45cc7e581088eae7d"}
May 25 14:24:59.950716 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.950420    2571 scope.go:117] "RemoveContainer" containerID="8894567ac68e20c6789c1d2d352339c44b32e2f0247c8be293aeecdb80d26948"
May 25 14:24:59.959157 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.959141    2571 scope.go:117] "RemoveContainer" containerID="a9aa842a386362d6b7f96d21a323bd5ed03f968f6f8f4423a95cc2d79fc29a95"
May 25 14:24:59.966149 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.966128    2571 scope.go:117] "RemoveContainer" containerID="b4e02781b5b7d4e909591c0e40648d0352a7181f24a5240576af02fc695e29c2"
May 25 14:24:59.972740 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.972723    2571 scope.go:117] "RemoveContainer" containerID="5be60284dcb878e0c63e3262ed62d1ac6afd36e9131260045ad19694c3c0bd1e"
May 25 14:24:59.978682 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.978660    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"]
May 25 14:24:59.979857 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.979844    2571 scope.go:117] "RemoveContainer" containerID="4affff872e8e34f001f75bfcdd229187395f80afed3416ca438c6cb88eeffc5a"
May 25 14:24:59.986818 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.986798    2571 scope.go:117] "RemoveContainer" containerID="813eea535a3542cd57f18d23d11d7c8a7c83a51c3b4b42e590cb08a7eeb969d1"
May 25 14:24:59.988601 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:24:59.988573    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-zzkbc"]
May 25 14:25:00.826403 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:00.826331    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28e27d47-f841-43ad-a928-814ca3435c37" path="/var/lib/kubelet/pods/28e27d47-f841-43ad-a928-814ca3435c37/volumes"
May 25 14:25:50.566270 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.566238    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"]
May 25 14:25:50.568710 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.566562    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="offline"
May 25 14:25:50.568710 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.566576    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="offline"
May 25 14:25:50.568710 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.566592    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="registry"
May 25 14:25:50.568710 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.566598    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="registry"
May 25 14:25:50.568710 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.566606    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="feast-init"
May 25 14:25:50.568710 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.566611    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="feast-init"
May 25 14:25:50.568710 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.566618    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="online"
May 25 14:25:50.568710 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.566623    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="online"
May 25 14:25:50.568710 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.566631    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="feast-apply"
May 25 14:25:50.568710 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.566637    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="feast-apply"
May 25 14:25:50.568710 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.566645    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="ui"
May 25 14:25:50.568710 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.566650    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="ui"
May 25 14:25:50.568710 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.566703    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="offline"
May 25 14:25:50.568710 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.566710    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="ui"
May 25 14:25:50.568710 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.566716    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="online"
May 25 14:25:50.568710 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.566724    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="28e27d47-f841-43ad-a928-814ca3435c37" containerName="registry"
May 25 14:25:50.569525 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.569510    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:25:50.571742 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.571709    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 25 14:25:50.571742 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.571717    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 25 14:25:50.572034 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.571792    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 25 14:25:50.572034 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.571739    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 25 14:25:50.572034 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.571739    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 25 14:25:50.572225 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.572210    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-tdqxl\""
May 25 14:25:50.572308 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.572217    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 25 14:25:50.579137 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.579118    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b46c7789-22d8-4a80-b53a-769aa79c46a9-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-dtj4n\" (UID: \"b46c7789-22d8-4a80-b53a-769aa79c46a9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:25:50.579240 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.579145    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b46c7789-22d8-4a80-b53a-769aa79c46a9-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-dtj4n\" (UID: \"b46c7789-22d8-4a80-b53a-769aa79c46a9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:25:50.579240 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.579161    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/b46c7789-22d8-4a80-b53a-769aa79c46a9-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-dtj4n\" (UID: \"b46c7789-22d8-4a80-b53a-769aa79c46a9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:25:50.579240 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.579185    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b46c7789-22d8-4a80-b53a-769aa79c46a9-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-dtj4n\" (UID: \"b46c7789-22d8-4a80-b53a-769aa79c46a9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:25:50.579390 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.579239    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b46c7789-22d8-4a80-b53a-769aa79c46a9-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-dtj4n\" (UID: \"b46c7789-22d8-4a80-b53a-769aa79c46a9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:25:50.579390 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.579261    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knn4c\" (UniqueName: \"kubernetes.io/projected/b46c7789-22d8-4a80-b53a-769aa79c46a9-kube-api-access-knn4c\") pod \"feast-simple-feast-remote-setup-5cff94746b-dtj4n\" (UID: \"b46c7789-22d8-4a80-b53a-769aa79c46a9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:25:50.586753 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.586730    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"]
May 25 14:25:50.679601 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.679575    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b46c7789-22d8-4a80-b53a-769aa79c46a9-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-dtj4n\" (UID: \"b46c7789-22d8-4a80-b53a-769aa79c46a9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:25:50.679779 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.679612    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b46c7789-22d8-4a80-b53a-769aa79c46a9-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-dtj4n\" (UID: \"b46c7789-22d8-4a80-b53a-769aa79c46a9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:25:50.679779 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.679639    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/b46c7789-22d8-4a80-b53a-769aa79c46a9-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-dtj4n\" (UID: \"b46c7789-22d8-4a80-b53a-769aa79c46a9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:25:50.679779 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.679666    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b46c7789-22d8-4a80-b53a-769aa79c46a9-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-dtj4n\" (UID: \"b46c7789-22d8-4a80-b53a-769aa79c46a9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:25:50.680029 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.679813    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b46c7789-22d8-4a80-b53a-769aa79c46a9-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-dtj4n\" (UID: \"b46c7789-22d8-4a80-b53a-769aa79c46a9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:25:50.680029 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.679860    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-knn4c\" (UniqueName: \"kubernetes.io/projected/b46c7789-22d8-4a80-b53a-769aa79c46a9-kube-api-access-knn4c\") pod \"feast-simple-feast-remote-setup-5cff94746b-dtj4n\" (UID: \"b46c7789-22d8-4a80-b53a-769aa79c46a9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:25:50.680132 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.680084    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b46c7789-22d8-4a80-b53a-769aa79c46a9-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-dtj4n\" (UID: \"b46c7789-22d8-4a80-b53a-769aa79c46a9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:25:50.680483 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.680457    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/b46c7789-22d8-4a80-b53a-769aa79c46a9-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-dtj4n\" (UID: \"b46c7789-22d8-4a80-b53a-769aa79c46a9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:25:50.682429 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.682403    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b46c7789-22d8-4a80-b53a-769aa79c46a9-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-dtj4n\" (UID: \"b46c7789-22d8-4a80-b53a-769aa79c46a9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:25:50.682522 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.682512    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b46c7789-22d8-4a80-b53a-769aa79c46a9-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-dtj4n\" (UID: \"b46c7789-22d8-4a80-b53a-769aa79c46a9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:25:50.682564 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.682530    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b46c7789-22d8-4a80-b53a-769aa79c46a9-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-dtj4n\" (UID: \"b46c7789-22d8-4a80-b53a-769aa79c46a9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:25:50.691264 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.691237    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knn4c\" (UniqueName: \"kubernetes.io/projected/b46c7789-22d8-4a80-b53a-769aa79c46a9-kube-api-access-knn4c\") pod \"feast-simple-feast-remote-setup-5cff94746b-dtj4n\" (UID: \"b46c7789-22d8-4a80-b53a-769aa79c46a9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:25:50.878907 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:50.878818    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:25:51.027919 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:51.027743    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"]
May 25 14:25:51.109667 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:51.109641    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n" event={"ID":"b46c7789-22d8-4a80-b53a-769aa79c46a9","Type":"ContainerStarted","Data":"e0b69327a336982b76b798b6a06b4eed2084e5969e57d4061643ddc7fd2f9314"}
May 25 14:25:51.109667 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:51.109672    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n" event={"ID":"b46c7789-22d8-4a80-b53a-769aa79c46a9","Type":"ContainerStarted","Data":"5aedb017ab2644573ec98376f6715814089e4cbdf5a2217b3892f7bfdde264e0"}
May 25 14:25:56.123744 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:56.123708    2571 generic.go:358] "Generic (PLEG): container finished" podID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerID="e0b69327a336982b76b798b6a06b4eed2084e5969e57d4061643ddc7fd2f9314" exitCode=0
May 25 14:25:56.124245 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:56.123763    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n" event={"ID":"b46c7789-22d8-4a80-b53a-769aa79c46a9","Type":"ContainerDied","Data":"e0b69327a336982b76b798b6a06b4eed2084e5969e57d4061643ddc7fd2f9314"}
May 25 14:25:57.134683 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:25:57.134643    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n" event={"ID":"b46c7789-22d8-4a80-b53a-769aa79c46a9","Type":"ContainerStarted","Data":"72ef952b0f5fe2d27bfe8805f3d323a746208d5d632e062eea38bb81d15abe3f"}
May 25 14:26:02.152748 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:26:02.152715    2571 generic.go:358] "Generic (PLEG): container finished" podID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerID="72ef952b0f5fe2d27bfe8805f3d323a746208d5d632e062eea38bb81d15abe3f" exitCode=0
May 25 14:26:02.153129 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:26:02.152789    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n" event={"ID":"b46c7789-22d8-4a80-b53a-769aa79c46a9","Type":"ContainerDied","Data":"72ef952b0f5fe2d27bfe8805f3d323a746208d5d632e062eea38bb81d15abe3f"}
May 25 14:26:03.158978 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:26:03.158918    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n" event={"ID":"b46c7789-22d8-4a80-b53a-769aa79c46a9","Type":"ContainerStarted","Data":"aca55ca6d9b532fdb0cae2544d7a0cb4e4224cb6dceffe7d3d2f13ee12f63dfd"}
May 25 14:26:03.159454 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:26:03.158993    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n" event={"ID":"b46c7789-22d8-4a80-b53a-769aa79c46a9","Type":"ContainerStarted","Data":"b3b7dcb191945ba8f1ee2d4828b71a847af2dd2a35b02534c3cd9272fc40f19f"}
May 25 14:26:03.159454 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:26:03.159010    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n" event={"ID":"b46c7789-22d8-4a80-b53a-769aa79c46a9","Type":"ContainerStarted","Data":"e25f187bec0da45a898d5c6aa11d56550050d10126b536f30ce4452259e1d9ca"}
May 25 14:26:03.199564 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:26:03.199430    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n" podStartSLOduration=13.199411488 podStartE2EDuration="13.199411488s" podCreationTimestamp="2026-05-25 14:25:50 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 14:26:03.197381236 +0000 UTC m=+1571.032237330" watchObservedRunningTime="2026-05-25 14:26:03.199411488 +0000 UTC m=+1571.034267582"
May 25 14:26:05.879782 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:26:05.879743    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:26:05.879782 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:26:05.879791    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:26:05.880341 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:26:05.879803    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:26:05.881438 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:26:05.881402    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n" podUID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.39:8816: connect: connection refused"
May 25 14:26:05.881585 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:26:05.881455    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n" podUID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerName="online" probeResult="failure" output="Get \"https://10.134.0.39:6567/health\": dial tcp 10.134.0.39:6567: connect: connection refused"
May 25 14:26:05.881585 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:26:05.881415    2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n" podUID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.39:8443: connect: connection refused"
May 25 14:26:08.879893 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:26:08.879858    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:26:08.880406 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:26:08.880361    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:26:08.880406 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:26:08.880394    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:26:08.880649 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:26:08.880632    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:26:08.884353 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:26:08.884336    2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:26:09.178761 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:26:09.178676    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:26:09.178761 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:26:09.178715    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:26:09.179083 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:26:09.179063    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:26:09.181777 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:26:09.181753    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:27:11.519406 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:11.519375    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"]
May 25 14:27:11.520022 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:11.519672    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n" podUID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerName="online" containerID="cri-o://e25f187bec0da45a898d5c6aa11d56550050d10126b536f30ce4452259e1d9ca" gracePeriod=30
May 25 14:27:11.520022 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:11.519759    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n" podUID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerName="offline" containerID="cri-o://b3b7dcb191945ba8f1ee2d4828b71a847af2dd2a35b02534c3cd9272fc40f19f" gracePeriod=30
May 25 14:27:11.520022 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:11.519756    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n" podUID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerName="ui" containerID="cri-o://aca55ca6d9b532fdb0cae2544d7a0cb4e4224cb6dceffe7d3d2f13ee12f63dfd" gracePeriod=30
May 25 14:27:11.663644 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:11.663612    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"]
May 25 14:27:11.663991 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:11.663959    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="registry" containerID="cri-o://1947ba2c9b3ebcd1b179a06a7a12a5a7c60075e6ba1801196a016e3ef124b3c0" gracePeriod=30
May 25 14:27:11.664129 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:11.664007    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="ui" containerID="cri-o://0887819ee016ef1655bf91444dfb19ff6a0f715af969e34d62db98707de50b6d" gracePeriod=30
May 25 14:27:11.664129 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:11.664013    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="online" containerID="cri-o://936ae754df4725e4959388ede119b921301d2d31747f235ec5fb830ca6579f6a" gracePeriod=30
May 25 14:27:11.664129 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:11.664038    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="offline" containerID="cri-o://21e7899f14412f683669a52882af11e5615dbadfca66875e396c71752e0b64d2" gracePeriod=30
May 25 14:27:12.376032 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:12.376004    2571 generic.go:358] "Generic (PLEG): container finished" podID="e62d45b9-1363-415f-be2a-8cec44d72278" containerID="0887819ee016ef1655bf91444dfb19ff6a0f715af969e34d62db98707de50b6d" exitCode=0
May 25 14:27:12.376131 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:12.376078    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" event={"ID":"e62d45b9-1363-415f-be2a-8cec44d72278","Type":"ContainerDied","Data":"0887819ee016ef1655bf91444dfb19ff6a0f715af969e34d62db98707de50b6d"}
May 25 14:27:12.378450 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:12.378426    2571 generic.go:358] "Generic (PLEG): container finished" podID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerID="aca55ca6d9b532fdb0cae2544d7a0cb4e4224cb6dceffe7d3d2f13ee12f63dfd" exitCode=0
May 25 14:27:12.378450 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:12.378448    2571 generic.go:358] "Generic (PLEG): container finished" podID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerID="e25f187bec0da45a898d5c6aa11d56550050d10126b536f30ce4452259e1d9ca" exitCode=0
May 25 14:27:12.378588 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:12.378497    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n" event={"ID":"b46c7789-22d8-4a80-b53a-769aa79c46a9","Type":"ContainerDied","Data":"aca55ca6d9b532fdb0cae2544d7a0cb4e4224cb6dceffe7d3d2f13ee12f63dfd"}
May 25 14:27:12.378588 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:12.378523    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n" event={"ID":"b46c7789-22d8-4a80-b53a-769aa79c46a9","Type":"ContainerDied","Data":"e25f187bec0da45a898d5c6aa11d56550050d10126b536f30ce4452259e1d9ca"}
May 25 14:27:13.384067 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:13.384037    2571 generic.go:358] "Generic (PLEG): container finished" podID="e62d45b9-1363-415f-be2a-8cec44d72278" containerID="936ae754df4725e4959388ede119b921301d2d31747f235ec5fb830ca6579f6a" exitCode=0
May 25 14:27:13.384435 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:13.384106    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" event={"ID":"e62d45b9-1363-415f-be2a-8cec44d72278","Type":"ContainerDied","Data":"936ae754df4725e4959388ede119b921301d2d31747f235ec5fb830ca6579f6a"}
May 25 14:27:18.103911 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:18.103872    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.38:8443: connect: connection refused"
May 25 14:27:18.881239 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:18.881197    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n" podUID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.39:8443: connect: connection refused"
May 25 14:27:18.912853 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:18.912821    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="online" probeResult="failure" output="Get \"https://10.134.0.38:6567/health\": dial tcp 10.134.0.38:6567: connect: connection refused"
May 25 14:27:19.179881 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:19.179784    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n" podUID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerName="online" probeResult="failure" output="Get \"https://10.134.0.39:6567/health\": dial tcp 10.134.0.39:6567: connect: connection refused"
May 25 14:27:28.103621 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:28.103573    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.38:8443: connect: connection refused"
May 25 14:27:28.880563 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:28.880524    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n" podUID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.39:8443: connect: connection refused"
May 25 14:27:28.912462 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:28.912416    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="online" probeResult="failure" output="Get \"https://10.134.0.38:6567/health\": dial tcp 10.134.0.38:6567: connect: connection refused"
May 25 14:27:29.179608 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:29.179502    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n" podUID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerName="online" probeResult="failure" output="Get \"https://10.134.0.39:6567/health\": dial tcp 10.134.0.39:6567: connect: connection refused"
May 25 14:27:38.103242 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:38.103196    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.38:8443: connect: connection refused"
May 25 14:27:38.103700 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:38.103358    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:27:38.880630 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:38.880593    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n" podUID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.39:8443: connect: connection refused"
May 25 14:27:38.880804 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:38.880696    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:27:38.912957 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:38.912903    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="online" probeResult="failure" output="Get \"https://10.134.0.38:6567/health\": dial tcp 10.134.0.38:6567: connect: connection refused"
May 25 14:27:38.913087 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:38.913021    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:27:39.179752 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:39.179658    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n" podUID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerName="online" probeResult="failure" output="Get \"https://10.134.0.39:6567/health\": dial tcp 10.134.0.39:6567: connect: connection refused"
May 25 14:27:39.180132 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:39.179762    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:27:41.811077 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:41.811055    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:27:41.981970 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:41.981925    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b46c7789-22d8-4a80-b53a-769aa79c46a9-ui-tls\") pod \"b46c7789-22d8-4a80-b53a-769aa79c46a9\" (UID: \"b46c7789-22d8-4a80-b53a-769aa79c46a9\") "
May 25 14:27:41.982101 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:41.981981    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/b46c7789-22d8-4a80-b53a-769aa79c46a9-registry-tls\") pod \"b46c7789-22d8-4a80-b53a-769aa79c46a9\" (UID: \"b46c7789-22d8-4a80-b53a-769aa79c46a9\") "
May 25 14:27:41.982101 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:41.982013    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b46c7789-22d8-4a80-b53a-769aa79c46a9-offline-tls\") pod \"b46c7789-22d8-4a80-b53a-769aa79c46a9\" (UID: \"b46c7789-22d8-4a80-b53a-769aa79c46a9\") "
May 25 14:27:41.982101 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:41.982051    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b46c7789-22d8-4a80-b53a-769aa79c46a9-feast-data\") pod \"b46c7789-22d8-4a80-b53a-769aa79c46a9\" (UID: \"b46c7789-22d8-4a80-b53a-769aa79c46a9\") "
May 25 14:27:41.982101 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:41.982068    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-knn4c\" (UniqueName: \"kubernetes.io/projected/b46c7789-22d8-4a80-b53a-769aa79c46a9-kube-api-access-knn4c\") pod \"b46c7789-22d8-4a80-b53a-769aa79c46a9\" (UID: \"b46c7789-22d8-4a80-b53a-769aa79c46a9\") "
May 25 14:27:41.982285 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:41.982110    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b46c7789-22d8-4a80-b53a-769aa79c46a9-online-tls\") pod \"b46c7789-22d8-4a80-b53a-769aa79c46a9\" (UID: \"b46c7789-22d8-4a80-b53a-769aa79c46a9\") "
May 25 14:27:41.982382 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:41.982351    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b46c7789-22d8-4a80-b53a-769aa79c46a9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b46c7789-22d8-4a80-b53a-769aa79c46a9" (UID: "b46c7789-22d8-4a80-b53a-769aa79c46a9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 14:27:41.983199 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:41.983162    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b46c7789-22d8-4a80-b53a-769aa79c46a9-feast-data" (OuterVolumeSpecName: "feast-data") pod "b46c7789-22d8-4a80-b53a-769aa79c46a9" (UID: "b46c7789-22d8-4a80-b53a-769aa79c46a9"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 14:27:41.984350 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:41.984318    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b46c7789-22d8-4a80-b53a-769aa79c46a9-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "b46c7789-22d8-4a80-b53a-769aa79c46a9" (UID: "b46c7789-22d8-4a80-b53a-769aa79c46a9"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:27:41.984454 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:41.984382    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b46c7789-22d8-4a80-b53a-769aa79c46a9-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "b46c7789-22d8-4a80-b53a-769aa79c46a9" (UID: "b46c7789-22d8-4a80-b53a-769aa79c46a9"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:27:41.984454 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:41.984385    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b46c7789-22d8-4a80-b53a-769aa79c46a9-kube-api-access-knn4c" (OuterVolumeSpecName: "kube-api-access-knn4c") pod "b46c7789-22d8-4a80-b53a-769aa79c46a9" (UID: "b46c7789-22d8-4a80-b53a-769aa79c46a9"). InnerVolumeSpecName "kube-api-access-knn4c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 14:27:41.984454 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:41.984397    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b46c7789-22d8-4a80-b53a-769aa79c46a9-online-tls" (OuterVolumeSpecName: "online-tls") pod "b46c7789-22d8-4a80-b53a-769aa79c46a9" (UID: "b46c7789-22d8-4a80-b53a-769aa79c46a9"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:27:42.083515 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.083461    2571 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b46c7789-22d8-4a80-b53a-769aa79c46a9-online-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:27:42.083515 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.083481    2571 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b46c7789-22d8-4a80-b53a-769aa79c46a9-ui-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:27:42.083515 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.083491    2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/b46c7789-22d8-4a80-b53a-769aa79c46a9-registry-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:27:42.083515 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.083499    2571 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b46c7789-22d8-4a80-b53a-769aa79c46a9-offline-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:27:42.083515 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.083506    2571 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b46c7789-22d8-4a80-b53a-769aa79c46a9-feast-data\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:27:42.083515 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.083514    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knn4c\" (UniqueName: \"kubernetes.io/projected/b46c7789-22d8-4a80-b53a-769aa79c46a9-kube-api-access-knn4c\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:27:42.305532 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.305511    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:27:42.386207 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.386144    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-offline-tls\") pod \"e62d45b9-1363-415f-be2a-8cec44d72278\" (UID: \"e62d45b9-1363-415f-be2a-8cec44d72278\") "
May 25 14:27:42.386207 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.386183    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-registry-tls\") pod \"e62d45b9-1363-415f-be2a-8cec44d72278\" (UID: \"e62d45b9-1363-415f-be2a-8cec44d72278\") "
May 25 14:27:42.386387 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.386210    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-ui-tls\") pod \"e62d45b9-1363-415f-be2a-8cec44d72278\" (UID: \"e62d45b9-1363-415f-be2a-8cec44d72278\") "
May 25 14:27:42.386387 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.386236    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-online-tls\") pod \"e62d45b9-1363-415f-be2a-8cec44d72278\" (UID: \"e62d45b9-1363-415f-be2a-8cec44d72278\") "
May 25 14:27:42.386387 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.386284    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgt66\" (UniqueName: \"kubernetes.io/projected/e62d45b9-1363-415f-be2a-8cec44d72278-kube-api-access-wgt66\") pod \"e62d45b9-1363-415f-be2a-8cec44d72278\" (UID: \"e62d45b9-1363-415f-be2a-8cec44d72278\") "
May 25 14:27:42.386387 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.386315    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e62d45b9-1363-415f-be2a-8cec44d72278-feast-data\") pod \"e62d45b9-1363-415f-be2a-8cec44d72278\" (UID: \"e62d45b9-1363-415f-be2a-8cec44d72278\") "
May 25 14:27:42.386879 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.386839    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e62d45b9-1363-415f-be2a-8cec44d72278-feast-data" (OuterVolumeSpecName: "feast-data") pod "e62d45b9-1363-415f-be2a-8cec44d72278" (UID: "e62d45b9-1363-415f-be2a-8cec44d72278"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 14:27:42.388639 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.388610    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e62d45b9-1363-415f-be2a-8cec44d72278-kube-api-access-wgt66" (OuterVolumeSpecName: "kube-api-access-wgt66") pod "e62d45b9-1363-415f-be2a-8cec44d72278" (UID: "e62d45b9-1363-415f-be2a-8cec44d72278"). InnerVolumeSpecName "kube-api-access-wgt66". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 14:27:42.389010 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.388984    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "e62d45b9-1363-415f-be2a-8cec44d72278" (UID: "e62d45b9-1363-415f-be2a-8cec44d72278"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:27:42.389117 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.389010    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e62d45b9-1363-415f-be2a-8cec44d72278" (UID: "e62d45b9-1363-415f-be2a-8cec44d72278"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:27:42.389117 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.389046    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-online-tls" (OuterVolumeSpecName: "online-tls") pod "e62d45b9-1363-415f-be2a-8cec44d72278" (UID: "e62d45b9-1363-415f-be2a-8cec44d72278"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:27:42.389117 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.389072    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "e62d45b9-1363-415f-be2a-8cec44d72278" (UID: "e62d45b9-1363-415f-be2a-8cec44d72278"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 14:27:42.475198 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.475167    2571 generic.go:358] "Generic (PLEG): container finished" podID="e62d45b9-1363-415f-be2a-8cec44d72278" containerID="21e7899f14412f683669a52882af11e5615dbadfca66875e396c71752e0b64d2" exitCode=137
May 25 14:27:42.475198 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.475189    2571 generic.go:358] "Generic (PLEG): container finished" podID="e62d45b9-1363-415f-be2a-8cec44d72278" containerID="1947ba2c9b3ebcd1b179a06a7a12a5a7c60075e6ba1801196a016e3ef124b3c0" exitCode=137
May 25 14:27:42.475412 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.475249    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" event={"ID":"e62d45b9-1363-415f-be2a-8cec44d72278","Type":"ContainerDied","Data":"21e7899f14412f683669a52882af11e5615dbadfca66875e396c71752e0b64d2"}
May 25 14:27:42.475412 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.475255    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"
May 25 14:27:42.475412 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.475288    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" event={"ID":"e62d45b9-1363-415f-be2a-8cec44d72278","Type":"ContainerDied","Data":"1947ba2c9b3ebcd1b179a06a7a12a5a7c60075e6ba1801196a016e3ef124b3c0"}
May 25 14:27:42.475412 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.475299    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb" event={"ID":"e62d45b9-1363-415f-be2a-8cec44d72278","Type":"ContainerDied","Data":"bc03ceed7141b5a45b9e814d2341b145e802489eb9e165ea9fd6bb6457f451ce"}
May 25 14:27:42.475412 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.475314    2571 scope.go:117] "RemoveContainer" containerID="0887819ee016ef1655bf91444dfb19ff6a0f715af969e34d62db98707de50b6d"
May 25 14:27:42.477733 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.477711    2571 generic.go:358] "Generic (PLEG): container finished" podID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerID="b3b7dcb191945ba8f1ee2d4828b71a847af2dd2a35b02534c3cd9272fc40f19f" exitCode=137
May 25 14:27:42.477824 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.477743    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n" event={"ID":"b46c7789-22d8-4a80-b53a-769aa79c46a9","Type":"ContainerDied","Data":"b3b7dcb191945ba8f1ee2d4828b71a847af2dd2a35b02534c3cd9272fc40f19f"}
May 25 14:27:42.477824 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.477782    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n" event={"ID":"b46c7789-22d8-4a80-b53a-769aa79c46a9","Type":"ContainerDied","Data":"5aedb017ab2644573ec98376f6715814089e4cbdf5a2217b3892f7bfdde264e0"}
May 25 14:27:42.477824 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.477794    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"
May 25 14:27:42.484018 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.483996    2571 scope.go:117] "RemoveContainer" containerID="21e7899f14412f683669a52882af11e5615dbadfca66875e396c71752e0b64d2"
May 25 14:27:42.487038 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.486863    2571 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-offline-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:27:42.487038 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.486884    2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-registry-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:27:42.487038 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.486898    2571 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-ui-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:27:42.487038 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.486908    2571 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e62d45b9-1363-415f-be2a-8cec44d72278-online-tls\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:27:42.487038 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.486917    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgt66\" (UniqueName: \"kubernetes.io/projected/e62d45b9-1363-415f-be2a-8cec44d72278-kube-api-access-wgt66\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:27:42.487038 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.486926    2571 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e62d45b9-1363-415f-be2a-8cec44d72278-feast-data\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:27:42.491501 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.491486    2571 scope.go:117] "RemoveContainer" containerID="936ae754df4725e4959388ede119b921301d2d31747f235ec5fb830ca6579f6a"
May 25 14:27:42.498395 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.498380    2571 scope.go:117] "RemoveContainer" containerID="1947ba2c9b3ebcd1b179a06a7a12a5a7c60075e6ba1801196a016e3ef124b3c0"
May 25 14:27:42.502509 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.502486    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"]
May 25 14:27:42.505502 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.505486    2571 scope.go:117] "RemoveContainer" containerID="d02f710874e38dac58faa4857320c8f6d0e87c751fc9d09399e924f223769fcc"
May 25 14:27:42.511907 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.511883    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-hspxb"]
May 25 14:27:42.512500 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.512484    2571 scope.go:117] "RemoveContainer" containerID="86f83e18ea9cc4d89ed3130a61186c42c90652543a5b70e7b4ed4528c31a9f06"
May 25 14:27:42.519452 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.519438    2571 scope.go:117] "RemoveContainer" containerID="0887819ee016ef1655bf91444dfb19ff6a0f715af969e34d62db98707de50b6d"
May 25 14:27:42.519688 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:27:42.519670    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0887819ee016ef1655bf91444dfb19ff6a0f715af969e34d62db98707de50b6d\": container with ID starting with 0887819ee016ef1655bf91444dfb19ff6a0f715af969e34d62db98707de50b6d not found: ID does not exist" containerID="0887819ee016ef1655bf91444dfb19ff6a0f715af969e34d62db98707de50b6d"
May 25 14:27:42.519751 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.519694    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0887819ee016ef1655bf91444dfb19ff6a0f715af969e34d62db98707de50b6d"} err="failed to get container status \"0887819ee016ef1655bf91444dfb19ff6a0f715af969e34d62db98707de50b6d\": rpc error: code = NotFound desc = could not find container \"0887819ee016ef1655bf91444dfb19ff6a0f715af969e34d62db98707de50b6d\": container with ID starting with 0887819ee016ef1655bf91444dfb19ff6a0f715af969e34d62db98707de50b6d not found: ID does not exist"
May 25 14:27:42.519751 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.519711    2571 scope.go:117] "RemoveContainer" containerID="21e7899f14412f683669a52882af11e5615dbadfca66875e396c71752e0b64d2"
May 25 14:27:42.519967 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:27:42.519927    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21e7899f14412f683669a52882af11e5615dbadfca66875e396c71752e0b64d2\": container with ID starting with 21e7899f14412f683669a52882af11e5615dbadfca66875e396c71752e0b64d2 not found: ID does not exist" containerID="21e7899f14412f683669a52882af11e5615dbadfca66875e396c71752e0b64d2"
May 25 14:27:42.520045 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.519977    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21e7899f14412f683669a52882af11e5615dbadfca66875e396c71752e0b64d2"} err="failed to get container status \"21e7899f14412f683669a52882af11e5615dbadfca66875e396c71752e0b64d2\": rpc error: code = NotFound desc = could not find container \"21e7899f14412f683669a52882af11e5615dbadfca66875e396c71752e0b64d2\": container with ID starting with 21e7899f14412f683669a52882af11e5615dbadfca66875e396c71752e0b64d2 not found: ID does not exist"
May 25 14:27:42.520045 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.520000    2571 scope.go:117] "RemoveContainer" containerID="936ae754df4725e4959388ede119b921301d2d31747f235ec5fb830ca6579f6a"
May 25 14:27:42.520257 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:27:42.520237    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"936ae754df4725e4959388ede119b921301d2d31747f235ec5fb830ca6579f6a\": container with ID starting with 936ae754df4725e4959388ede119b921301d2d31747f235ec5fb830ca6579f6a not found: ID does not exist" containerID="936ae754df4725e4959388ede119b921301d2d31747f235ec5fb830ca6579f6a"
May 25 14:27:42.520302 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.520260    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"936ae754df4725e4959388ede119b921301d2d31747f235ec5fb830ca6579f6a"} err="failed to get container status \"936ae754df4725e4959388ede119b921301d2d31747f235ec5fb830ca6579f6a\": rpc error: code = NotFound desc = could not find container \"936ae754df4725e4959388ede119b921301d2d31747f235ec5fb830ca6579f6a\": container with ID starting with 936ae754df4725e4959388ede119b921301d2d31747f235ec5fb830ca6579f6a not found: ID does not exist"
May 25 14:27:42.520302 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.520273    2571 scope.go:117] "RemoveContainer" containerID="1947ba2c9b3ebcd1b179a06a7a12a5a7c60075e6ba1801196a016e3ef124b3c0"
May 25 14:27:42.520501 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:27:42.520485    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1947ba2c9b3ebcd1b179a06a7a12a5a7c60075e6ba1801196a016e3ef124b3c0\": container with ID starting with 1947ba2c9b3ebcd1b179a06a7a12a5a7c60075e6ba1801196a016e3ef124b3c0 not found: ID does not exist" containerID="1947ba2c9b3ebcd1b179a06a7a12a5a7c60075e6ba1801196a016e3ef124b3c0"
May 25 14:27:42.520544 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.520506    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1947ba2c9b3ebcd1b179a06a7a12a5a7c60075e6ba1801196a016e3ef124b3c0"} err="failed to get container status \"1947ba2c9b3ebcd1b179a06a7a12a5a7c60075e6ba1801196a016e3ef124b3c0\": rpc error: code = NotFound desc = could not find container \"1947ba2c9b3ebcd1b179a06a7a12a5a7c60075e6ba1801196a016e3ef124b3c0\": container with ID starting with 1947ba2c9b3ebcd1b179a06a7a12a5a7c60075e6ba1801196a016e3ef124b3c0 not found: ID does not exist"
May 25 14:27:42.520544 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.520520    2571 scope.go:117] "RemoveContainer" containerID="d02f710874e38dac58faa4857320c8f6d0e87c751fc9d09399e924f223769fcc"
May 25 14:27:42.520743 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:27:42.520726    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d02f710874e38dac58faa4857320c8f6d0e87c751fc9d09399e924f223769fcc\": container with ID starting with d02f710874e38dac58faa4857320c8f6d0e87c751fc9d09399e924f223769fcc not found: ID does not exist" containerID="d02f710874e38dac58faa4857320c8f6d0e87c751fc9d09399e924f223769fcc"
May 25 14:27:42.520783 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.520747    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d02f710874e38dac58faa4857320c8f6d0e87c751fc9d09399e924f223769fcc"} err="failed to get container status \"d02f710874e38dac58faa4857320c8f6d0e87c751fc9d09399e924f223769fcc\": rpc error: code = NotFound desc = could not find container \"d02f710874e38dac58faa4857320c8f6d0e87c751fc9d09399e924f223769fcc\": container with ID starting with d02f710874e38dac58faa4857320c8f6d0e87c751fc9d09399e924f223769fcc not found: ID does not exist"
May 25 14:27:42.520783 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.520761    2571 scope.go:117] "RemoveContainer" containerID="86f83e18ea9cc4d89ed3130a61186c42c90652543a5b70e7b4ed4528c31a9f06"
May 25 14:27:42.521018 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:27:42.521001    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86f83e18ea9cc4d89ed3130a61186c42c90652543a5b70e7b4ed4528c31a9f06\": container with ID starting with 86f83e18ea9cc4d89ed3130a61186c42c90652543a5b70e7b4ed4528c31a9f06 not found: ID does not exist" containerID="86f83e18ea9cc4d89ed3130a61186c42c90652543a5b70e7b4ed4528c31a9f06"
May 25 14:27:42.521088 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.521021    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86f83e18ea9cc4d89ed3130a61186c42c90652543a5b70e7b4ed4528c31a9f06"} err="failed to get container status \"86f83e18ea9cc4d89ed3130a61186c42c90652543a5b70e7b4ed4528c31a9f06\": rpc error: code = NotFound desc = could not find container \"86f83e18ea9cc4d89ed3130a61186c42c90652543a5b70e7b4ed4528c31a9f06\": container with ID starting with 86f83e18ea9cc4d89ed3130a61186c42c90652543a5b70e7b4ed4528c31a9f06 not found: ID does not exist"
May 25 14:27:42.521088 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.521034    2571 scope.go:117] "RemoveContainer" containerID="0887819ee016ef1655bf91444dfb19ff6a0f715af969e34d62db98707de50b6d"
May 25 14:27:42.521261 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.521239    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0887819ee016ef1655bf91444dfb19ff6a0f715af969e34d62db98707de50b6d"} err="failed to get container status \"0887819ee016ef1655bf91444dfb19ff6a0f715af969e34d62db98707de50b6d\": rpc error: code = NotFound desc = could not find container \"0887819ee016ef1655bf91444dfb19ff6a0f715af969e34d62db98707de50b6d\": container with ID starting with 0887819ee016ef1655bf91444dfb19ff6a0f715af969e34d62db98707de50b6d not found: ID does not exist"
May 25 14:27:42.521311 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.521263    2571 scope.go:117] "RemoveContainer" containerID="21e7899f14412f683669a52882af11e5615dbadfca66875e396c71752e0b64d2"
May 25 14:27:42.521451 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.521432    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21e7899f14412f683669a52882af11e5615dbadfca66875e396c71752e0b64d2"} err="failed to get container status \"21e7899f14412f683669a52882af11e5615dbadfca66875e396c71752e0b64d2\": rpc error: code = NotFound desc = could not find container \"21e7899f14412f683669a52882af11e5615dbadfca66875e396c71752e0b64d2\": container with ID starting with 21e7899f14412f683669a52882af11e5615dbadfca66875e396c71752e0b64d2 not found: ID does not exist"
May 25 14:27:42.521497 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.521452    2571 scope.go:117] "RemoveContainer" containerID="936ae754df4725e4959388ede119b921301d2d31747f235ec5fb830ca6579f6a"
May 25 14:27:42.521639 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.521624    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"936ae754df4725e4959388ede119b921301d2d31747f235ec5fb830ca6579f6a"} err="failed to get container status \"936ae754df4725e4959388ede119b921301d2d31747f235ec5fb830ca6579f6a\": rpc error: code = NotFound desc = could not find container \"936ae754df4725e4959388ede119b921301d2d31747f235ec5fb830ca6579f6a\": container with ID starting with 936ae754df4725e4959388ede119b921301d2d31747f235ec5fb830ca6579f6a not found: ID does not exist"
May 25 14:27:42.521684 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.521639    2571 scope.go:117] "RemoveContainer" containerID="1947ba2c9b3ebcd1b179a06a7a12a5a7c60075e6ba1801196a016e3ef124b3c0"
May 25 14:27:42.521850 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.521829    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1947ba2c9b3ebcd1b179a06a7a12a5a7c60075e6ba1801196a016e3ef124b3c0"} err="failed to get container status \"1947ba2c9b3ebcd1b179a06a7a12a5a7c60075e6ba1801196a016e3ef124b3c0\": rpc error: code = NotFound desc = could not find container \"1947ba2c9b3ebcd1b179a06a7a12a5a7c60075e6ba1801196a016e3ef124b3c0\": container with ID starting with 1947ba2c9b3ebcd1b179a06a7a12a5a7c60075e6ba1801196a016e3ef124b3c0 not found: ID does not exist"
May 25 14:27:42.521898 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.521851    2571 scope.go:117] "RemoveContainer" containerID="d02f710874e38dac58faa4857320c8f6d0e87c751fc9d09399e924f223769fcc"
May 25 14:27:42.522057 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.522040    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d02f710874e38dac58faa4857320c8f6d0e87c751fc9d09399e924f223769fcc"} err="failed to get container status \"d02f710874e38dac58faa4857320c8f6d0e87c751fc9d09399e924f223769fcc\": rpc error: code = NotFound desc = could not find container \"d02f710874e38dac58faa4857320c8f6d0e87c751fc9d09399e924f223769fcc\": container with ID starting with d02f710874e38dac58faa4857320c8f6d0e87c751fc9d09399e924f223769fcc not found: ID does not exist"
May 25 14:27:42.522105 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.522058    2571 scope.go:117] "RemoveContainer" containerID="86f83e18ea9cc4d89ed3130a61186c42c90652543a5b70e7b4ed4528c31a9f06"
May 25 14:27:42.522243 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.522228    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86f83e18ea9cc4d89ed3130a61186c42c90652543a5b70e7b4ed4528c31a9f06"} err="failed to get container status \"86f83e18ea9cc4d89ed3130a61186c42c90652543a5b70e7b4ed4528c31a9f06\": rpc error: code = NotFound desc = could not find container \"86f83e18ea9cc4d89ed3130a61186c42c90652543a5b70e7b4ed4528c31a9f06\": container with ID starting with 86f83e18ea9cc4d89ed3130a61186c42c90652543a5b70e7b4ed4528c31a9f06 not found: ID does not exist"
May 25 14:27:42.522285 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.522244    2571 scope.go:117] "RemoveContainer" containerID="aca55ca6d9b532fdb0cae2544d7a0cb4e4224cb6dceffe7d3d2f13ee12f63dfd"
May 25 14:27:42.524640 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.524622    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"]
May 25 14:27:42.529546 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.529535    2571 scope.go:117] "RemoveContainer" containerID="b3b7dcb191945ba8f1ee2d4828b71a847af2dd2a35b02534c3cd9272fc40f19f"
May 25 14:27:42.535067 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.535050    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-dtj4n"]
May 25 14:27:42.536635 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.536624    2571 scope.go:117] "RemoveContainer" containerID="e25f187bec0da45a898d5c6aa11d56550050d10126b536f30ce4452259e1d9ca"
May 25 14:27:42.552499 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.552477    2571 scope.go:117] "RemoveContainer" containerID="72ef952b0f5fe2d27bfe8805f3d323a746208d5d632e062eea38bb81d15abe3f"
May 25 14:27:42.558920 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.558905    2571 scope.go:117] "RemoveContainer" containerID="e0b69327a336982b76b798b6a06b4eed2084e5969e57d4061643ddc7fd2f9314"
May 25 14:27:42.565492 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.565477    2571 scope.go:117] "RemoveContainer" containerID="aca55ca6d9b532fdb0cae2544d7a0cb4e4224cb6dceffe7d3d2f13ee12f63dfd"
May 25 14:27:42.565722 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:27:42.565704    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aca55ca6d9b532fdb0cae2544d7a0cb4e4224cb6dceffe7d3d2f13ee12f63dfd\": container with ID starting with aca55ca6d9b532fdb0cae2544d7a0cb4e4224cb6dceffe7d3d2f13ee12f63dfd not found: ID does not exist" containerID="aca55ca6d9b532fdb0cae2544d7a0cb4e4224cb6dceffe7d3d2f13ee12f63dfd"
May 25 14:27:42.565795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.565726    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aca55ca6d9b532fdb0cae2544d7a0cb4e4224cb6dceffe7d3d2f13ee12f63dfd"} err="failed to get container status \"aca55ca6d9b532fdb0cae2544d7a0cb4e4224cb6dceffe7d3d2f13ee12f63dfd\": rpc error: code = NotFound desc = could not find container \"aca55ca6d9b532fdb0cae2544d7a0cb4e4224cb6dceffe7d3d2f13ee12f63dfd\": container with ID starting with aca55ca6d9b532fdb0cae2544d7a0cb4e4224cb6dceffe7d3d2f13ee12f63dfd not found: ID does not exist"
May 25 14:27:42.565795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.565748    2571 scope.go:117] "RemoveContainer" containerID="b3b7dcb191945ba8f1ee2d4828b71a847af2dd2a35b02534c3cd9272fc40f19f"
May 25 14:27:42.565983 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:27:42.565957    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3b7dcb191945ba8f1ee2d4828b71a847af2dd2a35b02534c3cd9272fc40f19f\": container with ID starting with b3b7dcb191945ba8f1ee2d4828b71a847af2dd2a35b02534c3cd9272fc40f19f not found: ID does not exist" containerID="b3b7dcb191945ba8f1ee2d4828b71a847af2dd2a35b02534c3cd9272fc40f19f"
May 25 14:27:42.566069 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.565986    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3b7dcb191945ba8f1ee2d4828b71a847af2dd2a35b02534c3cd9272fc40f19f"} err="failed to get container status \"b3b7dcb191945ba8f1ee2d4828b71a847af2dd2a35b02534c3cd9272fc40f19f\": rpc error: code = NotFound desc = could not find container \"b3b7dcb191945ba8f1ee2d4828b71a847af2dd2a35b02534c3cd9272fc40f19f\": container with ID starting with b3b7dcb191945ba8f1ee2d4828b71a847af2dd2a35b02534c3cd9272fc40f19f not found: ID does not exist"
May 25 14:27:42.566069 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.566010    2571 scope.go:117] "RemoveContainer" containerID="e25f187bec0da45a898d5c6aa11d56550050d10126b536f30ce4452259e1d9ca"
May 25 14:27:42.566231 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:27:42.566216    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e25f187bec0da45a898d5c6aa11d56550050d10126b536f30ce4452259e1d9ca\": container with ID starting with e25f187bec0da45a898d5c6aa11d56550050d10126b536f30ce4452259e1d9ca not found: ID does not exist" containerID="e25f187bec0da45a898d5c6aa11d56550050d10126b536f30ce4452259e1d9ca"
May 25 14:27:42.566297 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.566237    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e25f187bec0da45a898d5c6aa11d56550050d10126b536f30ce4452259e1d9ca"} err="failed to get container status \"e25f187bec0da45a898d5c6aa11d56550050d10126b536f30ce4452259e1d9ca\": rpc error: code = NotFound desc = could not find container \"e25f187bec0da45a898d5c6aa11d56550050d10126b536f30ce4452259e1d9ca\": container with ID starting with e25f187bec0da45a898d5c6aa11d56550050d10126b536f30ce4452259e1d9ca not found: ID does not exist"
May 25 14:27:42.566297 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.566250    2571 scope.go:117] "RemoveContainer" containerID="72ef952b0f5fe2d27bfe8805f3d323a746208d5d632e062eea38bb81d15abe3f"
May 25 14:27:42.566455 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:27:42.566440    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72ef952b0f5fe2d27bfe8805f3d323a746208d5d632e062eea38bb81d15abe3f\": container with ID starting with 72ef952b0f5fe2d27bfe8805f3d323a746208d5d632e062eea38bb81d15abe3f not found: ID does not exist" containerID="72ef952b0f5fe2d27bfe8805f3d323a746208d5d632e062eea38bb81d15abe3f"
May 25 14:27:42.566496 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.566457    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72ef952b0f5fe2d27bfe8805f3d323a746208d5d632e062eea38bb81d15abe3f"} err="failed to get container status \"72ef952b0f5fe2d27bfe8805f3d323a746208d5d632e062eea38bb81d15abe3f\": rpc error: code = NotFound desc = could not find container \"72ef952b0f5fe2d27bfe8805f3d323a746208d5d632e062eea38bb81d15abe3f\": container with ID starting with 72ef952b0f5fe2d27bfe8805f3d323a746208d5d632e062eea38bb81d15abe3f not found: ID does not exist"
May 25 14:27:42.566496 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.566467    2571 scope.go:117] "RemoveContainer" containerID="e0b69327a336982b76b798b6a06b4eed2084e5969e57d4061643ddc7fd2f9314"
May 25 14:27:42.566689 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:27:42.566674    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0b69327a336982b76b798b6a06b4eed2084e5969e57d4061643ddc7fd2f9314\": container with ID starting with e0b69327a336982b76b798b6a06b4eed2084e5969e57d4061643ddc7fd2f9314 not found: ID does not exist" containerID="e0b69327a336982b76b798b6a06b4eed2084e5969e57d4061643ddc7fd2f9314"
May 25 14:27:42.566746 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.566692    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0b69327a336982b76b798b6a06b4eed2084e5969e57d4061643ddc7fd2f9314"} err="failed to get container status \"e0b69327a336982b76b798b6a06b4eed2084e5969e57d4061643ddc7fd2f9314\": rpc error: code = NotFound desc = could not find container \"e0b69327a336982b76b798b6a06b4eed2084e5969e57d4061643ddc7fd2f9314\": container with ID starting with e0b69327a336982b76b798b6a06b4eed2084e5969e57d4061643ddc7fd2f9314 not found: ID does not exist"
May 25 14:27:42.825342 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.825312    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b46c7789-22d8-4a80-b53a-769aa79c46a9" path="/var/lib/kubelet/pods/b46c7789-22d8-4a80-b53a-769aa79c46a9/volumes"
May 25 14:27:42.825864 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:27:42.825850    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" path="/var/lib/kubelet/pods/e62d45b9-1363-415f-be2a-8cec44d72278/volumes"
May 25 14:28:01.748387 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748315    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mlj9r/must-gather-97r28"]
May 25 14:28:01.748795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748613    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="offline"
May 25 14:28:01.748795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748623    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="offline"
May 25 14:28:01.748795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748631    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="registry"
May 25 14:28:01.748795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748637    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="registry"
May 25 14:28:01.748795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748649    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerName="ui"
May 25 14:28:01.748795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748655    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerName="ui"
May 25 14:28:01.748795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748661    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="online"
May 25 14:28:01.748795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748666    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="online"
May 25 14:28:01.748795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748671    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerName="feast-init"
May 25 14:28:01.748795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748677    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerName="feast-init"
May 25 14:28:01.748795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748686    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="feast-init"
May 25 14:28:01.748795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748690    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="feast-init"
May 25 14:28:01.748795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748701    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerName="offline"
May 25 14:28:01.748795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748705    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerName="offline"
May 25 14:28:01.748795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748712    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerName="feast-apply"
May 25 14:28:01.748795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748718    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerName="feast-apply"
May 25 14:28:01.748795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748723    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerName="online"
May 25 14:28:01.748795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748728    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerName="online"
May 25 14:28:01.748795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748734    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="feast-apply"
May 25 14:28:01.748795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748738    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="feast-apply"
May 25 14:28:01.748795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748743    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="ui"
May 25 14:28:01.748795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748749    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="ui"
May 25 14:28:01.748795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748792    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="offline"
May 25 14:28:01.748795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748800    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerName="ui"
May 25 14:28:01.749512 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748808    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerName="online"
May 25 14:28:01.749512 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748814    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="b46c7789-22d8-4a80-b53a-769aa79c46a9" containerName="offline"
May 25 14:28:01.749512 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748820    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="online"
May 25 14:28:01.749512 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748827    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="registry"
May 25 14:28:01.749512 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.748834    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="e62d45b9-1363-415f-be2a-8cec44d72278" containerName="ui"
May 25 14:28:01.753231 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.753211    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mlj9r/must-gather-97r28"
May 25 14:28:01.755071 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.755047    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mlj9r\"/\"openshift-service-ca.crt\""
May 25 14:28:01.755589 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.755573    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mlj9r\"/\"kube-root-ca.crt\""
May 25 14:28:01.755662 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.755573    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mlj9r\"/\"default-dockercfg-7lv4x\""
May 25 14:28:01.757776 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.757753    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mlj9r/must-gather-97r28"]
May 25 14:28:01.830317 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.830290    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dxh5\" (UniqueName: \"kubernetes.io/projected/805deca3-e5ff-4b7c-934c-30a1389c6ae6-kube-api-access-9dxh5\") pod \"must-gather-97r28\" (UID: \"805deca3-e5ff-4b7c-934c-30a1389c6ae6\") " pod="openshift-must-gather-mlj9r/must-gather-97r28"
May 25 14:28:01.830426 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.830406    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/805deca3-e5ff-4b7c-934c-30a1389c6ae6-must-gather-output\") pod \"must-gather-97r28\" (UID: \"805deca3-e5ff-4b7c-934c-30a1389c6ae6\") " pod="openshift-must-gather-mlj9r/must-gather-97r28"
May 25 14:28:01.931333 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.931305    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9dxh5\" (UniqueName: \"kubernetes.io/projected/805deca3-e5ff-4b7c-934c-30a1389c6ae6-kube-api-access-9dxh5\") pod \"must-gather-97r28\" (UID: \"805deca3-e5ff-4b7c-934c-30a1389c6ae6\") " pod="openshift-must-gather-mlj9r/must-gather-97r28"
May 25 14:28:01.931436 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.931344    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/805deca3-e5ff-4b7c-934c-30a1389c6ae6-must-gather-output\") pod \"must-gather-97r28\" (UID: \"805deca3-e5ff-4b7c-934c-30a1389c6ae6\") " pod="openshift-must-gather-mlj9r/must-gather-97r28"
May 25 14:28:01.931650 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.931634    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/805deca3-e5ff-4b7c-934c-30a1389c6ae6-must-gather-output\") pod \"must-gather-97r28\" (UID: \"805deca3-e5ff-4b7c-934c-30a1389c6ae6\") " pod="openshift-must-gather-mlj9r/must-gather-97r28"
May 25 14:28:01.939247 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:01.939229    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dxh5\" (UniqueName: \"kubernetes.io/projected/805deca3-e5ff-4b7c-934c-30a1389c6ae6-kube-api-access-9dxh5\") pod \"must-gather-97r28\" (UID: \"805deca3-e5ff-4b7c-934c-30a1389c6ae6\") " pod="openshift-must-gather-mlj9r/must-gather-97r28"
May 25 14:28:02.071411 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:02.071353    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mlj9r/must-gather-97r28"
May 25 14:28:02.189058 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:02.188923    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mlj9r/must-gather-97r28"]
May 25 14:28:02.191710 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:28:02.191687    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod805deca3_e5ff_4b7c_934c_30a1389c6ae6.slice/crio-d84257c5a987e536c64baf1b83fa23205c51ab026cf3101f49693251b8508a58 WatchSource:0}: Error finding container d84257c5a987e536c64baf1b83fa23205c51ab026cf3101f49693251b8508a58: Status 404 returned error can't find the container with id d84257c5a987e536c64baf1b83fa23205c51ab026cf3101f49693251b8508a58
May 25 14:28:02.551483 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:02.551448    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mlj9r/must-gather-97r28" event={"ID":"805deca3-e5ff-4b7c-934c-30a1389c6ae6","Type":"ContainerStarted","Data":"d84257c5a987e536c64baf1b83fa23205c51ab026cf3101f49693251b8508a58"}
May 25 14:28:08.572039 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:08.571912    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mlj9r/must-gather-97r28" event={"ID":"805deca3-e5ff-4b7c-934c-30a1389c6ae6","Type":"ContainerStarted","Data":"2828bf3db853e0cbd3a334629e89498e9596506ee1342cdaeca4ead0137375ba"}
May 25 14:28:08.572039 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:08.571991    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mlj9r/must-gather-97r28" event={"ID":"805deca3-e5ff-4b7c-934c-30a1389c6ae6","Type":"ContainerStarted","Data":"e32dc8e94edd551b346840ac58b2cb43d3482a41dba788723d1c098afff8f266"}
May 25 14:28:08.589024 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:08.588962    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-mlj9r/must-gather-97r28" podStartSLOduration=1.562070549 podStartE2EDuration="7.588922094s" podCreationTimestamp="2026-05-25 14:28:01 +0000 UTC" firstStartedPulling="2026-05-25 14:28:02.193360874 +0000 UTC m=+1690.028216947" lastFinishedPulling="2026-05-25 14:28:08.220212418 +0000 UTC m=+1696.055068492" observedRunningTime="2026-05-25 14:28:08.588097286 +0000 UTC m=+1696.422953379" watchObservedRunningTime="2026-05-25 14:28:08.588922094 +0000 UTC m=+1696.423778191"
May 25 14:28:18.603169 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:18.603132    2571 generic.go:358] "Generic (PLEG): container finished" podID="805deca3-e5ff-4b7c-934c-30a1389c6ae6" containerID="e32dc8e94edd551b346840ac58b2cb43d3482a41dba788723d1c098afff8f266" exitCode=0
May 25 14:28:18.603564 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:18.603189    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mlj9r/must-gather-97r28" event={"ID":"805deca3-e5ff-4b7c-934c-30a1389c6ae6","Type":"ContainerDied","Data":"e32dc8e94edd551b346840ac58b2cb43d3482a41dba788723d1c098afff8f266"}
May 25 14:28:18.603564 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:18.603476    2571 scope.go:117] "RemoveContainer" containerID="e32dc8e94edd551b346840ac58b2cb43d3482a41dba788723d1c098afff8f266"
May 25 14:28:18.729975 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:18.729927    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-mlj9r_must-gather-97r28_805deca3-e5ff-4b7c-934c-30a1389c6ae6/gather/0.log"
May 25 14:28:21.813870 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:21.813842    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6w2rv_0f731cec-0664-4e49-891b-53bd4603dbba/global-pull-secret-syncer/0.log"
May 25 14:28:21.996927 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:21.996899    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-c8rbn_a0a2025d-c91e-4fd7-a82a-37a9416c9fc5/konnectivity-agent/0.log"
May 25 14:28:22.103075 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:22.103010    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-220.ec2.internal_9707757169265407340417083d330c50/haproxy/0.log"
May 25 14:28:24.068698 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.068663    2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-mlj9r/must-gather-97r28"]
May 25 14:28:24.069191 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.068884    2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-must-gather-mlj9r/must-gather-97r28" podUID="805deca3-e5ff-4b7c-934c-30a1389c6ae6" containerName="copy" containerID="cri-o://2828bf3db853e0cbd3a334629e89498e9596506ee1342cdaeca4ead0137375ba" gracePeriod=2
May 25 14:28:24.073863 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.073839    2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-mlj9r/must-gather-97r28"]
May 25 14:28:24.297203 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.297184    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-mlj9r_must-gather-97r28_805deca3-e5ff-4b7c-934c-30a1389c6ae6/copy/0.log"
May 25 14:28:24.297544 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.297530    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mlj9r/must-gather-97r28"
May 25 14:28:24.299330 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.299302    2571 status_manager.go:895] "Failed to get status for pod" podUID="805deca3-e5ff-4b7c-934c-30a1389c6ae6" pod="openshift-must-gather-mlj9r/must-gather-97r28" err="pods \"must-gather-97r28\" is forbidden: User \"system:node:ip-10-0-128-220.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-mlj9r\": no relationship found between node 'ip-10-0-128-220.ec2.internal' and this object"
May 25 14:28:24.424602 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.424538    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/805deca3-e5ff-4b7c-934c-30a1389c6ae6-must-gather-output\") pod \"805deca3-e5ff-4b7c-934c-30a1389c6ae6\" (UID: \"805deca3-e5ff-4b7c-934c-30a1389c6ae6\") "
May 25 14:28:24.424602 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.424593    2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9dxh5\" (UniqueName: \"kubernetes.io/projected/805deca3-e5ff-4b7c-934c-30a1389c6ae6-kube-api-access-9dxh5\") pod \"805deca3-e5ff-4b7c-934c-30a1389c6ae6\" (UID: \"805deca3-e5ff-4b7c-934c-30a1389c6ae6\") "
May 25 14:28:24.424876 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.424853    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/805deca3-e5ff-4b7c-934c-30a1389c6ae6-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "805deca3-e5ff-4b7c-934c-30a1389c6ae6" (UID: "805deca3-e5ff-4b7c-934c-30a1389c6ae6"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 14:28:24.426836 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.426817    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/805deca3-e5ff-4b7c-934c-30a1389c6ae6-kube-api-access-9dxh5" (OuterVolumeSpecName: "kube-api-access-9dxh5") pod "805deca3-e5ff-4b7c-934c-30a1389c6ae6" (UID: "805deca3-e5ff-4b7c-934c-30a1389c6ae6"). InnerVolumeSpecName "kube-api-access-9dxh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 14:28:24.525112 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.525086    2571 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/805deca3-e5ff-4b7c-934c-30a1389c6ae6-must-gather-output\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:28:24.525112 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.525108    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dxh5\" (UniqueName: \"kubernetes.io/projected/805deca3-e5ff-4b7c-934c-30a1389c6ae6-kube-api-access-9dxh5\") on node \"ip-10-0-128-220.ec2.internal\" DevicePath \"\""
May 25 14:28:24.621472 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.621452    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-mlj9r_must-gather-97r28_805deca3-e5ff-4b7c-934c-30a1389c6ae6/copy/0.log"
May 25 14:28:24.621810 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.621789    2571 generic.go:358] "Generic (PLEG): container finished" podID="805deca3-e5ff-4b7c-934c-30a1389c6ae6" containerID="2828bf3db853e0cbd3a334629e89498e9596506ee1342cdaeca4ead0137375ba" exitCode=143
May 25 14:28:24.621883 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.621842    2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mlj9r/must-gather-97r28"
May 25 14:28:24.621883 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.621875    2571 scope.go:117] "RemoveContainer" containerID="2828bf3db853e0cbd3a334629e89498e9596506ee1342cdaeca4ead0137375ba"
May 25 14:28:24.623810 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.623777    2571 status_manager.go:895] "Failed to get status for pod" podUID="805deca3-e5ff-4b7c-934c-30a1389c6ae6" pod="openshift-must-gather-mlj9r/must-gather-97r28" err="pods \"must-gather-97r28\" is forbidden: User \"system:node:ip-10-0-128-220.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-mlj9r\": no relationship found between node 'ip-10-0-128-220.ec2.internal' and this object"
May 25 14:28:24.629061 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.629005    2571 scope.go:117] "RemoveContainer" containerID="e32dc8e94edd551b346840ac58b2cb43d3482a41dba788723d1c098afff8f266"
May 25 14:28:24.631258 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.631232    2571 status_manager.go:895] "Failed to get status for pod" podUID="805deca3-e5ff-4b7c-934c-30a1389c6ae6" pod="openshift-must-gather-mlj9r/must-gather-97r28" err="pods \"must-gather-97r28\" is forbidden: User \"system:node:ip-10-0-128-220.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-mlj9r\": no relationship found between node 'ip-10-0-128-220.ec2.internal' and this object"
May 25 14:28:24.639599 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.639583    2571 scope.go:117] "RemoveContainer" containerID="2828bf3db853e0cbd3a334629e89498e9596506ee1342cdaeca4ead0137375ba"
May 25 14:28:24.639876 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:28:24.639860    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2828bf3db853e0cbd3a334629e89498e9596506ee1342cdaeca4ead0137375ba\": container with ID starting with 2828bf3db853e0cbd3a334629e89498e9596506ee1342cdaeca4ead0137375ba not found: ID does not exist" containerID="2828bf3db853e0cbd3a334629e89498e9596506ee1342cdaeca4ead0137375ba"
May 25 14:28:24.639913 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.639887    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2828bf3db853e0cbd3a334629e89498e9596506ee1342cdaeca4ead0137375ba"} err="failed to get container status \"2828bf3db853e0cbd3a334629e89498e9596506ee1342cdaeca4ead0137375ba\": rpc error: code = NotFound desc = could not find container \"2828bf3db853e0cbd3a334629e89498e9596506ee1342cdaeca4ead0137375ba\": container with ID starting with 2828bf3db853e0cbd3a334629e89498e9596506ee1342cdaeca4ead0137375ba not found: ID does not exist"
May 25 14:28:24.639913 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.639905    2571 scope.go:117] "RemoveContainer" containerID="e32dc8e94edd551b346840ac58b2cb43d3482a41dba788723d1c098afff8f266"
May 25 14:28:24.640173 ip-10-0-128-220 kubenswrapper[2571]: E0525 14:28:24.640156    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e32dc8e94edd551b346840ac58b2cb43d3482a41dba788723d1c098afff8f266\": container with ID starting with e32dc8e94edd551b346840ac58b2cb43d3482a41dba788723d1c098afff8f266 not found: ID does not exist" containerID="e32dc8e94edd551b346840ac58b2cb43d3482a41dba788723d1c098afff8f266"
May 25 14:28:24.640215 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.640180    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e32dc8e94edd551b346840ac58b2cb43d3482a41dba788723d1c098afff8f266"} err="failed to get container status \"e32dc8e94edd551b346840ac58b2cb43d3482a41dba788723d1c098afff8f266\": rpc error: code = NotFound desc = could not find container \"e32dc8e94edd551b346840ac58b2cb43d3482a41dba788723d1c098afff8f266\": container with ID starting with e32dc8e94edd551b346840ac58b2cb43d3482a41dba788723d1c098afff8f266 not found: ID does not exist"
May 25 14:28:24.712379 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.712352    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f28e36a1-7faa-4ece-b092-aebf4ea1f218/alertmanager/0.log"
May 25 14:28:24.746092 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.746065    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f28e36a1-7faa-4ece-b092-aebf4ea1f218/config-reloader/0.log"
May 25 14:28:24.798712 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.798690    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f28e36a1-7faa-4ece-b092-aebf4ea1f218/kube-rbac-proxy-web/0.log"
May 25 14:28:24.825738 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.825715    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="805deca3-e5ff-4b7c-934c-30a1389c6ae6" path="/var/lib/kubelet/pods/805deca3-e5ff-4b7c-934c-30a1389c6ae6/volumes"
May 25 14:28:24.837706 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.837686    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f28e36a1-7faa-4ece-b092-aebf4ea1f218/kube-rbac-proxy/0.log"
May 25 14:28:24.869748 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.869729    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f28e36a1-7faa-4ece-b092-aebf4ea1f218/kube-rbac-proxy-metric/0.log"
May 25 14:28:24.914520 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.914499    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f28e36a1-7faa-4ece-b092-aebf4ea1f218/prom-label-proxy/0.log"
May 25 14:28:24.948334 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:24.948315    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f28e36a1-7faa-4ece-b092-aebf4ea1f218/init-config-reloader/0.log"
May 25 14:28:25.037898 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:25.037838    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-nvqq7_75bd5dfa-11be-44fe-b4da-b5c06ddee30a/kube-state-metrics/0.log"
May 25 14:28:25.070503 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:25.070484    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-nvqq7_75bd5dfa-11be-44fe-b4da-b5c06ddee30a/kube-rbac-proxy-main/0.log"
May 25 14:28:25.106184 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:25.106165    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-nvqq7_75bd5dfa-11be-44fe-b4da-b5c06ddee30a/kube-rbac-proxy-self/0.log"
May 25 14:28:25.150403 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:25.150376    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-65898b7c8b-68bw5_95adfa22-3ac3-4d92-8c86-165fcb12a609/metrics-server/0.log"
May 25 14:28:25.199777 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:25.199757    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-665c9c8584-xlxsr_d5521ae8-eced-4397-8e19-ae7b75b147e5/monitoring-plugin/0.log"
May 25 14:28:25.403407 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:25.403340    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dnnst_7e2be626-b053-44b8-b177-c43e4e451265/node-exporter/0.log"
May 25 14:28:25.435795 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:25.435774    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dnnst_7e2be626-b053-44b8-b177-c43e4e451265/kube-rbac-proxy/0.log"
May 25 14:28:25.471170 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:25.471152    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dnnst_7e2be626-b053-44b8-b177-c43e4e451265/init-textfile/0.log"
May 25 14:28:25.621035 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:25.621011    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-rxqh2_77b90865-975f-46ae-bb1f-e7cf87aef32a/kube-rbac-proxy-main/0.log"
May 25 14:28:25.665983 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:25.665885    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-rxqh2_77b90865-975f-46ae-bb1f-e7cf87aef32a/kube-rbac-proxy-self/0.log"
May 25 14:28:25.704264 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:25.704244    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-rxqh2_77b90865-975f-46ae-bb1f-e7cf87aef32a/openshift-state-metrics/0.log"
May 25 14:28:25.757568 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:25.757475    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ee10d117-ba7d-43c6-9669-79951557238f/prometheus/0.log"
May 25 14:28:25.799670 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:25.799649    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ee10d117-ba7d-43c6-9669-79951557238f/config-reloader/0.log"
May 25 14:28:25.843467 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:25.843446    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ee10d117-ba7d-43c6-9669-79951557238f/thanos-sidecar/0.log"
May 25 14:28:25.892099 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:25.892081    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ee10d117-ba7d-43c6-9669-79951557238f/kube-rbac-proxy-web/0.log"
May 25 14:28:25.933332 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:25.933265    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ee10d117-ba7d-43c6-9669-79951557238f/kube-rbac-proxy/0.log"
May 25 14:28:25.962991 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:25.962967    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ee10d117-ba7d-43c6-9669-79951557238f/kube-rbac-proxy-thanos/0.log"
May 25 14:28:26.001199 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:26.001176    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ee10d117-ba7d-43c6-9669-79951557238f/init-config-reloader/0.log"
May 25 14:28:26.197630 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:26.197605    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5489765fd6-qklxx_bbfe26d4-cd97-40c4-ab84-aeb546c3f07f/telemeter-client/0.log"
May 25 14:28:26.239048 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:26.239025    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5489765fd6-qklxx_bbfe26d4-cd97-40c4-ab84-aeb546c3f07f/reload/0.log"
May 25 14:28:26.283651 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:26.283629    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5489765fd6-qklxx_bbfe26d4-cd97-40c4-ab84-aeb546c3f07f/kube-rbac-proxy/0.log"
May 25 14:28:26.361823 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:26.361799    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f69c45485-shw4q_95f37b20-7603-4a8a-9e85-72c406d458fa/thanos-query/0.log"
May 25 14:28:26.425437 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:26.425411    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f69c45485-shw4q_95f37b20-7603-4a8a-9e85-72c406d458fa/kube-rbac-proxy-web/0.log"
May 25 14:28:26.496662 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:26.496634    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f69c45485-shw4q_95f37b20-7603-4a8a-9e85-72c406d458fa/kube-rbac-proxy/0.log"
May 25 14:28:26.541726 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:26.541706    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f69c45485-shw4q_95f37b20-7603-4a8a-9e85-72c406d458fa/prom-label-proxy/0.log"
May 25 14:28:26.579482 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:26.579462    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f69c45485-shw4q_95f37b20-7603-4a8a-9e85-72c406d458fa/kube-rbac-proxy-rules/0.log"
May 25 14:28:26.619515 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:26.619497    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f69c45485-shw4q_95f37b20-7603-4a8a-9e85-72c406d458fa/kube-rbac-proxy-metrics/0.log"
May 25 14:28:28.329016 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.328986    2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dvv4p/perf-node-gather-daemonset-nv84s"]
May 25 14:28:28.329500 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.329480    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="805deca3-e5ff-4b7c-934c-30a1389c6ae6" containerName="copy"
May 25 14:28:28.329576 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.329503    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="805deca3-e5ff-4b7c-934c-30a1389c6ae6" containerName="copy"
May 25 14:28:28.329576 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.329534    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="805deca3-e5ff-4b7c-934c-30a1389c6ae6" containerName="gather"
May 25 14:28:28.329576 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.329544    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="805deca3-e5ff-4b7c-934c-30a1389c6ae6" containerName="gather"
May 25 14:28:28.329719 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.329631    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="805deca3-e5ff-4b7c-934c-30a1389c6ae6" containerName="gather"
May 25 14:28:28.329719 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.329647    2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="805deca3-e5ff-4b7c-934c-30a1389c6ae6" containerName="copy"
May 25 14:28:28.332951 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.332914    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dvv4p/perf-node-gather-daemonset-nv84s"
May 25 14:28:28.335597 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.335567    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dvv4p\"/\"openshift-service-ca.crt\""
May 25 14:28:28.336144 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.336128    2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dvv4p\"/\"default-dockercfg-n9kjt\""
May 25 14:28:28.336240 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.336167    2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dvv4p\"/\"kube-root-ca.crt\""
May 25 14:28:28.343947 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.343910    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dvv4p/perf-node-gather-daemonset-nv84s"]
May 25 14:28:28.455699 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.455661    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25998831-1581-47b2-87aa-d713ae965318-lib-modules\") pod \"perf-node-gather-daemonset-nv84s\" (UID: \"25998831-1581-47b2-87aa-d713ae965318\") " pod="openshift-must-gather-dvv4p/perf-node-gather-daemonset-nv84s"
May 25 14:28:28.455839 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.455716    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25998831-1581-47b2-87aa-d713ae965318-sys\") pod \"perf-node-gather-daemonset-nv84s\" (UID: \"25998831-1581-47b2-87aa-d713ae965318\") " pod="openshift-must-gather-dvv4p/perf-node-gather-daemonset-nv84s"
May 25 14:28:28.455839 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.455743    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/25998831-1581-47b2-87aa-d713ae965318-podres\") pod \"perf-node-gather-daemonset-nv84s\" (UID: \"25998831-1581-47b2-87aa-d713ae965318\") " pod="openshift-must-gather-dvv4p/perf-node-gather-daemonset-nv84s"
May 25 14:28:28.455839 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.455771    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/25998831-1581-47b2-87aa-d713ae965318-proc\") pod \"perf-node-gather-daemonset-nv84s\" (UID: \"25998831-1581-47b2-87aa-d713ae965318\") " pod="openshift-must-gather-dvv4p/perf-node-gather-daemonset-nv84s"
May 25 14:28:28.455839 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.455807    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r25rs\" (UniqueName: \"kubernetes.io/projected/25998831-1581-47b2-87aa-d713ae965318-kube-api-access-r25rs\") pod \"perf-node-gather-daemonset-nv84s\" (UID: \"25998831-1581-47b2-87aa-d713ae965318\") " pod="openshift-must-gather-dvv4p/perf-node-gather-daemonset-nv84s"
May 25 14:28:28.556601 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.556580    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25998831-1581-47b2-87aa-d713ae965318-sys\") pod \"perf-node-gather-daemonset-nv84s\" (UID: \"25998831-1581-47b2-87aa-d713ae965318\") " pod="openshift-must-gather-dvv4p/perf-node-gather-daemonset-nv84s"
May 25 14:28:28.556686 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.556607    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/25998831-1581-47b2-87aa-d713ae965318-podres\") pod \"perf-node-gather-daemonset-nv84s\" (UID: \"25998831-1581-47b2-87aa-d713ae965318\") " pod="openshift-must-gather-dvv4p/perf-node-gather-daemonset-nv84s"
May 25 14:28:28.556686 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.556628    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/25998831-1581-47b2-87aa-d713ae965318-proc\") pod \"perf-node-gather-daemonset-nv84s\" (UID: \"25998831-1581-47b2-87aa-d713ae965318\") " pod="openshift-must-gather-dvv4p/perf-node-gather-daemonset-nv84s"
May 25 14:28:28.556686 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.556653    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r25rs\" (UniqueName: \"kubernetes.io/projected/25998831-1581-47b2-87aa-d713ae965318-kube-api-access-r25rs\") pod \"perf-node-gather-daemonset-nv84s\" (UID: \"25998831-1581-47b2-87aa-d713ae965318\") " pod="openshift-must-gather-dvv4p/perf-node-gather-daemonset-nv84s"
May 25 14:28:28.556842 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.556707    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25998831-1581-47b2-87aa-d713ae965318-sys\") pod \"perf-node-gather-daemonset-nv84s\" (UID: \"25998831-1581-47b2-87aa-d713ae965318\") " pod="openshift-must-gather-dvv4p/perf-node-gather-daemonset-nv84s"
May 25 14:28:28.556842 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.556720    2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25998831-1581-47b2-87aa-d713ae965318-lib-modules\") pod \"perf-node-gather-daemonset-nv84s\" (UID: \"25998831-1581-47b2-87aa-d713ae965318\") " pod="openshift-must-gather-dvv4p/perf-node-gather-daemonset-nv84s"
May 25 14:28:28.556842 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.556734    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/25998831-1581-47b2-87aa-d713ae965318-proc\") pod \"perf-node-gather-daemonset-nv84s\" (UID: \"25998831-1581-47b2-87aa-d713ae965318\") " pod="openshift-must-gather-dvv4p/perf-node-gather-daemonset-nv84s"
May 25 14:28:28.556842 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.556772    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/25998831-1581-47b2-87aa-d713ae965318-podres\") pod \"perf-node-gather-daemonset-nv84s\" (UID: \"25998831-1581-47b2-87aa-d713ae965318\") " pod="openshift-must-gather-dvv4p/perf-node-gather-daemonset-nv84s"
May 25 14:28:28.556842 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.556801    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25998831-1581-47b2-87aa-d713ae965318-lib-modules\") pod \"perf-node-gather-daemonset-nv84s\" (UID: \"25998831-1581-47b2-87aa-d713ae965318\") " pod="openshift-must-gather-dvv4p/perf-node-gather-daemonset-nv84s"
May 25 14:28:28.569693 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.569671    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r25rs\" (UniqueName: \"kubernetes.io/projected/25998831-1581-47b2-87aa-d713ae965318-kube-api-access-r25rs\") pod \"perf-node-gather-daemonset-nv84s\" (UID: \"25998831-1581-47b2-87aa-d713ae965318\") " pod="openshift-must-gather-dvv4p/perf-node-gather-daemonset-nv84s"
May 25 14:28:28.642757 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.642697    2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dvv4p/perf-node-gather-daemonset-nv84s"
May 25 14:28:28.772978 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.772952    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7947896bbc-j8l66_df3be7c8-3d64-4712-a3ee-d9d7c4ba98b5/console/0.log"
May 25 14:28:28.779093 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.779068    2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dvv4p/perf-node-gather-daemonset-nv84s"]
May 25 14:28:28.782397 ip-10-0-128-220 kubenswrapper[2571]: W0525 14:28:28.782374    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod25998831_1581_47b2_87aa_d713ae965318.slice/crio-44f2c4644ca5a9dae623c09bd3f7fc69a6d68be052c6a6e7fe1c5ce41dbee177 WatchSource:0}: Error finding container 44f2c4644ca5a9dae623c09bd3f7fc69a6d68be052c6a6e7fe1c5ce41dbee177: Status 404 returned error can't find the container with id 44f2c4644ca5a9dae623c09bd3f7fc69a6d68be052c6a6e7fe1c5ce41dbee177
May 25 14:28:28.783862 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:28.783840    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 25 14:28:29.639178 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:29.639140    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dvv4p/perf-node-gather-daemonset-nv84s" event={"ID":"25998831-1581-47b2-87aa-d713ae965318","Type":"ContainerStarted","Data":"4008e513d1f650f972eb94c2f60e58ef027d205241a9653fc382f881e8fb9be6"}
May 25 14:28:29.639178 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:29.639173    2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dvv4p/perf-node-gather-daemonset-nv84s" event={"ID":"25998831-1581-47b2-87aa-d713ae965318","Type":"ContainerStarted","Data":"44f2c4644ca5a9dae623c09bd3f7fc69a6d68be052c6a6e7fe1c5ce41dbee177"}
May 25 14:28:29.639579 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:29.639262    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dvv4p/perf-node-gather-daemonset-nv84s"
May 25 14:28:29.665577 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:29.665535    2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-dvv4p/perf-node-gather-daemonset-nv84s" podStartSLOduration=1.665520186 podStartE2EDuration="1.665520186s" podCreationTimestamp="2026-05-25 14:28:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 14:28:29.663172884 +0000 UTC m=+1717.498028976" watchObservedRunningTime="2026-05-25 14:28:29.665520186 +0000 UTC m=+1717.500376278"
May 25 14:28:30.214981 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:30.214927    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jqs89_68592d9d-e82d-4b8f-9be9-572afeaf463c/dns/0.log"
May 25 14:28:30.251305 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:30.251280    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jqs89_68592d9d-e82d-4b8f-9be9-572afeaf463c/kube-rbac-proxy/0.log"
May 25 14:28:30.474412 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:30.474341    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rx92j_68251ce2-60d3-4c72-8e3f-58d0ea9d9629/dns-node-resolver/0.log"
May 25 14:28:31.007695 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:31.007666    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7jmvx_370bcb80-f526-48ae-8582-1d104d764c96/node-ca/0.log"
May 25 14:28:32.230129 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:32.230100    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sn7fl_678dec39-f2e8-486d-9f09-56a72f3cdd85/serve-healthcheck-canary/0.log"
May 25 14:28:32.631673 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:32.631597    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5kgzp_24931dfe-9faa-4088-99c8-84ccb73daad2/kube-rbac-proxy/0.log"
May 25 14:28:32.656090 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:32.656066    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5kgzp_24931dfe-9faa-4088-99c8-84ccb73daad2/exporter/0.log"
May 25 14:28:32.682438 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:32.682410    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5kgzp_24931dfe-9faa-4088-99c8-84ccb73daad2/extractor/0.log"
May 25 14:28:35.653082 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:35.653049    2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dvv4p/perf-node-gather-daemonset-nv84s"
May 25 14:28:38.731093 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:38.731063    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-msrg9_dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f/kube-multus-additional-cni-plugins/0.log"
May 25 14:28:38.755898 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:38.755870    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-msrg9_dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f/egress-router-binary-copy/0.log"
May 25 14:28:38.783693 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:38.783668    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-msrg9_dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f/cni-plugins/0.log"
May 25 14:28:38.812096 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:38.812078    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-msrg9_dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f/bond-cni-plugin/0.log"
May 25 14:28:38.840187 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:38.840165    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-msrg9_dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f/routeoverride-cni/0.log"
May 25 14:28:38.869620 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:38.869599    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-msrg9_dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f/whereabouts-cni-bincopy/0.log"
May 25 14:28:38.901032 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:38.901013    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-msrg9_dbd6c6f2-ddd2-44f7-8ba0-11dbdeec293f/whereabouts-cni/0.log"
May 25 14:28:39.200916 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:39.200883    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-zm52l_6e393b7c-9ac9-4848-95f7-7aa7845d67d9/kube-multus/0.log"
May 25 14:28:39.322630 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:39.322593    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lc2jb_f9f60fa8-15a3-402e-842e-f1a52b7a3404/network-metrics-daemon/0.log"
May 25 14:28:39.344541 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:39.344520    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lc2jb_f9f60fa8-15a3-402e-842e-f1a52b7a3404/kube-rbac-proxy/0.log"
May 25 14:28:40.783551 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:40.783523    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fstv5_c7a16d3f-06d4-4855-bd37-0e1df89d84b5/ovn-controller/0.log"
May 25 14:28:40.831282 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:40.831250    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fstv5_c7a16d3f-06d4-4855-bd37-0e1df89d84b5/ovn-acl-logging/0.log"
May 25 14:28:40.853194 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:40.853170    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fstv5_c7a16d3f-06d4-4855-bd37-0e1df89d84b5/kube-rbac-proxy-node/0.log"
May 25 14:28:40.877901 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:40.877876    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fstv5_c7a16d3f-06d4-4855-bd37-0e1df89d84b5/kube-rbac-proxy-ovn-metrics/0.log"
May 25 14:28:40.906685 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:40.906640    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fstv5_c7a16d3f-06d4-4855-bd37-0e1df89d84b5/northd/0.log"
May 25 14:28:40.935402 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:40.935381    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fstv5_c7a16d3f-06d4-4855-bd37-0e1df89d84b5/nbdb/0.log"
May 25 14:28:40.962276 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:40.962260    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fstv5_c7a16d3f-06d4-4855-bd37-0e1df89d84b5/sbdb/0.log"
May 25 14:28:41.145634 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:41.145548    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fstv5_c7a16d3f-06d4-4855-bd37-0e1df89d84b5/ovnkube-controller/0.log"
May 25 14:28:42.445177 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:42.445147    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-fb2gw_98f00859-da06-410b-8572-c64348ab32d5/network-check-target-container/0.log"
May 25 14:28:43.329572 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:43.329541    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-bt7c9_198a8047-e88d-4bc2-805d-0bfd6155fb0f/iptables-alerter/0.log"
May 25 14:28:44.126667 ip-10-0-128-220 kubenswrapper[2571]: I0525 14:28:44.126635    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-bw5st_e3d0b072-8bba-48c2-98cc-d14341527f7c/tuned/0.log"