May 29 02:52:29.112237 ip-10-0-137-122 systemd[1]: Starting Kubernetes Kubelet...
May 29 02:52:29.615685 ip-10-0-137-122 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 29 02:52:29.615685 ip-10-0-137-122 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 29 02:52:29.615685 ip-10-0-137-122 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 29 02:52:29.615685 ip-10-0-137-122 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 29 02:52:29.615685 ip-10-0-137-122 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 29 02:52:29.617559 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.617476    2559 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 29 02:52:29.622356 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622342    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 29 02:52:29.622356 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622356    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 29 02:52:29.622423 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622360    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
May 29 02:52:29.622423 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622363    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 29 02:52:29.622423 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622366    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 29 02:52:29.622423 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622369    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 29 02:52:29.622423 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622372    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 29 02:52:29.622423 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622375    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 29 02:52:29.622423 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622377    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 29 02:52:29.622423 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622380    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 29 02:52:29.622423 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622382    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 29 02:52:29.622423 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622385    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 29 02:52:29.622423 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622388    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 29 02:52:29.622423 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622391    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 29 02:52:29.622423 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622394    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 29 02:52:29.622423 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622397    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 29 02:52:29.622423 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622400    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 29 02:52:29.622423 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622403    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 29 02:52:29.622423 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622406    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 29 02:52:29.622423 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622408    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 29 02:52:29.622423 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622411    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 29 02:52:29.622423 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622414    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 29 02:52:29.623072 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622417    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 29 02:52:29.623072 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622419    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 29 02:52:29.623072 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622422    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 29 02:52:29.623072 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622424    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 29 02:52:29.623072 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622427    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 29 02:52:29.623072 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622430    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 29 02:52:29.623072 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622433    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 29 02:52:29.623072 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622436    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 29 02:52:29.623072 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622438    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 29 02:52:29.623072 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622441    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 29 02:52:29.623072 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622443    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 29 02:52:29.623072 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622446    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 29 02:52:29.623072 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622448    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 29 02:52:29.623072 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622451    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 29 02:52:29.623072 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622453    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 29 02:52:29.623072 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622456    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 29 02:52:29.623072 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622458    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 29 02:52:29.623072 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622461    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 29 02:52:29.623072 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622464    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 29 02:52:29.623072 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622466    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 29 02:52:29.623708 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622469    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 29 02:52:29.623708 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622471    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 29 02:52:29.623708 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622474    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 29 02:52:29.623708 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622476    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
May 29 02:52:29.623708 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622479    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 29 02:52:29.623708 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622481    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 29 02:52:29.623708 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622484    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 29 02:52:29.623708 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622487    2559 feature_gate.go:328] unrecognized feature gate: Example
May 29 02:52:29.623708 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622490    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 29 02:52:29.623708 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622493    2559 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 29 02:52:29.623708 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622495    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 29 02:52:29.623708 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622498    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 29 02:52:29.623708 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622501    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 29 02:52:29.623708 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622505    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 29 02:52:29.623708 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622507    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 29 02:52:29.623708 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622510    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 29 02:52:29.623708 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622514    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 29 02:52:29.623708 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622518    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 29 02:52:29.623708 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622521    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 29 02:52:29.624202 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622525    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 29 02:52:29.624202 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622528    2559 feature_gate.go:328] unrecognized feature gate: Example2
May 29 02:52:29.624202 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622532    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 29 02:52:29.624202 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622535    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 29 02:52:29.624202 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622537    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 29 02:52:29.624202 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622540    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 29 02:52:29.624202 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622543    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 29 02:52:29.624202 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622546    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 29 02:52:29.624202 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622549    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 29 02:52:29.624202 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622551    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 29 02:52:29.624202 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622554    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 29 02:52:29.624202 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622556    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 29 02:52:29.624202 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622559    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 29 02:52:29.624202 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622562    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 29 02:52:29.624202 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622575    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 29 02:52:29.624202 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622578    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 29 02:52:29.624202 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622582    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 29 02:52:29.624202 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622584    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 29 02:52:29.624202 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622587    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 29 02:52:29.624656 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622590    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 29 02:52:29.624656 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622618    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 29 02:52:29.624656 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622655    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 29 02:52:29.624656 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622658    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 29 02:52:29.624656 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622662    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 29 02:52:29.624656 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.622665    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 29 02:52:29.625395 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625380    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 29 02:52:29.625395 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625394    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 29 02:52:29.625452 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625398    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 29 02:52:29.625452 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625401    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 29 02:52:29.625452 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625404    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 29 02:52:29.625452 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625409    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 29 02:52:29.625452 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625412    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 29 02:52:29.625452 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625415    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 29 02:52:29.625452 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625418    2559 feature_gate.go:328] unrecognized feature gate: Example2
May 29 02:52:29.625452 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625420    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 29 02:52:29.625452 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625423    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 29 02:52:29.625452 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625426    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 29 02:52:29.625452 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625428    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 29 02:52:29.625452 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625431    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 29 02:52:29.625452 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625433    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 29 02:52:29.625452 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625436    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 29 02:52:29.625452 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625438    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 29 02:52:29.625452 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625441    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 29 02:52:29.625452 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625444    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 29 02:52:29.625452 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625447    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 29 02:52:29.625452 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625450    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 29 02:52:29.625900 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625453    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 29 02:52:29.625900 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625456    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 29 02:52:29.625900 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625459    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 29 02:52:29.625900 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625461    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 29 02:52:29.625900 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625464    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 29 02:52:29.625900 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625467    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 29 02:52:29.625900 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625470    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 29 02:52:29.625900 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625473    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 29 02:52:29.625900 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625475    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 29 02:52:29.625900 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625478    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 29 02:52:29.625900 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625480    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 29 02:52:29.625900 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625483    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 29 02:52:29.625900 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625485    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 29 02:52:29.625900 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625488    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 29 02:52:29.625900 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625490    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 29 02:52:29.625900 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625493    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 29 02:52:29.625900 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625495    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 29 02:52:29.625900 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625498    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 29 02:52:29.625900 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625500    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 29 02:52:29.626382 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625503    2559 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 29 02:52:29.626382 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625505    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 29 02:52:29.626382 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625508    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 29 02:52:29.626382 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625511    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 29 02:52:29.626382 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625514    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 29 02:52:29.626382 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625517    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 29 02:52:29.626382 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625519    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 29 02:52:29.626382 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625522    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
May 29 02:52:29.626382 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625525    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 29 02:52:29.626382 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625527    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 29 02:52:29.626382 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625530    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 29 02:52:29.626382 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625533    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 29 02:52:29.626382 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625535    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 29 02:52:29.626382 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625538    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 29 02:52:29.626382 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625541    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 29 02:52:29.626382 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625544    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 29 02:52:29.626382 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625546    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 29 02:52:29.626382 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625549    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 29 02:52:29.626382 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625551    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 29 02:52:29.626382 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625554    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 29 02:52:29.626902 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625557    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
May 29 02:52:29.626902 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625559    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 29 02:52:29.626902 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625562    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 29 02:52:29.626902 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625564    2559 feature_gate.go:328] unrecognized feature gate: Example
May 29 02:52:29.626902 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625571    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 29 02:52:29.626902 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625574    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 29 02:52:29.626902 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625576    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 29 02:52:29.626902 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625579    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 29 02:52:29.626902 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625581    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 29 02:52:29.626902 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625583    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 29 02:52:29.626902 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625586    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 29 02:52:29.626902 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625588    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 29 02:52:29.626902 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625592    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 29 02:52:29.626902 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625597    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 29 02:52:29.626902 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625600    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 29 02:52:29.626902 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625602    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 29 02:52:29.626902 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625606    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 29 02:52:29.626902 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625608    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 29 02:52:29.626902 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625611    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 29 02:52:29.626902 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625614    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 29 02:52:29.627405 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625617    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 29 02:52:29.627405 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625619    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 29 02:52:29.627405 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625622    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 29 02:52:29.627405 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625625    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 29 02:52:29.627405 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625627    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 29 02:52:29.627405 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.625630    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 29 02:52:29.627405 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625697    2559 flags.go:64] FLAG: --address="0.0.0.0"
May 29 02:52:29.627405 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625704    2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 29 02:52:29.627405 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625710    2559 flags.go:64] FLAG: --anonymous-auth="true"
May 29 02:52:29.627405 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625714    2559 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 29 02:52:29.627405 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625719    2559 flags.go:64] FLAG: --authentication-token-webhook="false"
May 29 02:52:29.627405 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625722    2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 29 02:52:29.627405 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625726    2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 29 02:52:29.627405 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625730    2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 29 02:52:29.627405 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625733    2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 29 02:52:29.627405 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625736    2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 29 02:52:29.627405 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625740    2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 29 02:52:29.627405 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625743    2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 29 02:52:29.627405 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625746    2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 29 02:52:29.627405 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625749    2559 flags.go:64] FLAG: --cgroup-root=""
May 29 02:52:29.627405 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625752    2559 flags.go:64] FLAG: --cgroups-per-qos="true"
May 29 02:52:29.627405 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625755    2559 flags.go:64] FLAG: --client-ca-file=""
May 29 02:52:29.627946 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625758    2559 flags.go:64] FLAG: --cloud-config=""
May 29 02:52:29.627946 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625761    2559 flags.go:64] FLAG: --cloud-provider="external"
May 29 02:52:29.627946 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625764    2559 flags.go:64] FLAG: --cluster-dns="[]"
May 29 02:52:29.627946 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625770    2559 flags.go:64] FLAG: --cluster-domain=""
May 29 02:52:29.627946 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625773    2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 29 02:52:29.627946 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625776    2559 flags.go:64] FLAG: --config-dir=""
May 29 02:52:29.627946 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625779    2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 29 02:52:29.627946 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625782    2559 flags.go:64] FLAG: --container-log-max-files="5"
May 29 02:52:29.627946 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625786    2559 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 29 02:52:29.627946 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625789    2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 29 02:52:29.627946 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625792    2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 29 02:52:29.627946 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625796    2559 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 29 02:52:29.627946 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625799    2559 flags.go:64] FLAG: --contention-profiling="false"
May 29 02:52:29.627946 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625802    2559 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 29 02:52:29.627946 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625804    2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 29 02:52:29.627946 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625808    2559 flags.go:64] FLAG: --cpu-manager-policy="none"
May 29 02:52:29.627946 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625811    2559 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 29 02:52:29.627946 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625815    2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 29 02:52:29.627946 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625818    2559 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 29 02:52:29.627946 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625822    2559 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 29 02:52:29.627946 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625825    2559 flags.go:64] FLAG: --enable-load-reader="false"
May 29 02:52:29.627946 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625828    2559 flags.go:64] FLAG: --enable-server="true"
May 29 02:52:29.627946 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625831    2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 29 02:52:29.627946 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625835    2559 flags.go:64] FLAG: --event-burst="100"
May 29 02:52:29.627946 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625838    2559 flags.go:64] FLAG: --event-qps="50"
May 29 02:52:29.628558 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625841    2559 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 29 02:52:29.628558 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625844    2559 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 29 02:52:29.628558 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625846    2559 flags.go:64] FLAG: --eviction-hard=""
May 29 02:52:29.628558 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625850    2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 29 02:52:29.628558 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625853    2559 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 29 02:52:29.628558 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625856    2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 29 02:52:29.628558 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625859    2559 flags.go:64] FLAG: --eviction-soft=""
May 29 02:52:29.628558 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625862    2559 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 29 02:52:29.628558 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625865    2559 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 29 02:52:29.628558 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625868    2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 29 02:52:29.628558 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625871    2559 flags.go:64] FLAG: --experimental-mounter-path=""
May 29 02:52:29.628558 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625874    2559 flags.go:64] FLAG: --fail-cgroupv1="false"
May 29 02:52:29.628558 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625877    2559 flags.go:64] FLAG: --fail-swap-on="true"
May 29 02:52:29.628558 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625880    2559 flags.go:64] FLAG: --feature-gates=""
May 29 02:52:29.628558 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625884    2559 flags.go:64] FLAG: --file-check-frequency="20s"
May 29 02:52:29.628558 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625887    2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 29 02:52:29.628558 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625890    2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 29 02:52:29.628558 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625894    2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 29 02:52:29.628558 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625897    2559 flags.go:64] FLAG: --healthz-port="10248"
May 29 02:52:29.628558 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625900    2559 flags.go:64] FLAG: --help="false"
May 29 02:52:29.628558 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625903    2559 flags.go:64] FLAG: --hostname-override="ip-10-0-137-122.ec2.internal"
May 29 02:52:29.628558 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625906    2559 flags.go:64] FLAG: --housekeeping-interval="10s"
May 29 02:52:29.628558 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625910    2559 flags.go:64] FLAG: --http-check-frequency="20s"
May 29 02:52:29.628558 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625913    2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 29 02:52:29.629149 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625917    2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 29 02:52:29.629149 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625920    2559 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 29 02:52:29.629149 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625923    2559 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 29 02:52:29.629149 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625926    2559 flags.go:64] FLAG: --image-service-endpoint=""
May 29 02:52:29.629149 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625929    2559 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 29 02:52:29.629149 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625932    2559 flags.go:64] FLAG: --kube-api-burst="100"
May 29 02:52:29.629149 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625935    2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 29 02:52:29.629149 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625938    2559 flags.go:64] FLAG: --kube-api-qps="50"
May 29 02:52:29.629149 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625941    2559 flags.go:64] FLAG: --kube-reserved=""
May 29 02:52:29.629149 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625944    2559 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 29 02:52:29.629149 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625946    2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 29 02:52:29.629149 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625949    2559 flags.go:64] FLAG: --kubelet-cgroups=""
May 29 02:52:29.629149 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625952    2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 29 02:52:29.629149 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625955    2559 flags.go:64] FLAG: --lock-file=""
May 29 02:52:29.629149 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625958    2559 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 29 02:52:29.629149 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625961    2559 flags.go:64] FLAG: --log-flush-frequency="5s"
May 29 02:52:29.629149 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625977    2559 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 29 02:52:29.629149 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625983    2559 flags.go:64] FLAG: --log-json-split-stream="false"
May 29 02:52:29.629149 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625986    2559 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 29 02:52:29.629149 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625989    2559 flags.go:64] FLAG: --log-text-split-stream="false"
May 29 02:52:29.629149 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625992    2559 flags.go:64] FLAG: --logging-format="text"
May 29 02:52:29.629149 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625995    2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 29 02:52:29.629149 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.625999    2559 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 29 02:52:29.629149 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626002    2559 flags.go:64] FLAG: --manifest-url=""
May 29 02:52:29.629740 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626005    2559 flags.go:64] FLAG: --manifest-url-header=""
May 29 02:52:29.629740 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626010    2559 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 29 02:52:29.629740 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626013    2559 flags.go:64] FLAG: --max-open-files="1000000"
May 29 02:52:29.629740 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626016    2559 flags.go:64] FLAG: --max-pods="110"
May 29 02:52:29.629740 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626019    2559 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 29 02:52:29.629740 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626022    2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 29 02:52:29.629740 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626026    2559 flags.go:64] FLAG: --memory-manager-policy="None"
May 29 02:52:29.629740 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626029    2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 29 02:52:29.629740 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626032    2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 29 02:52:29.629740 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626035    2559 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 29 02:52:29.629740 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626038    2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 29 02:52:29.629740 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626045    2559 flags.go:64] FLAG: --node-status-max-images="50"
May 29 02:52:29.629740 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626048    2559 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 29 02:52:29.629740 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626051    2559 flags.go:64] FLAG: --oom-score-adj="-999"
May 29 02:52:29.629740 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626054    2559 flags.go:64] FLAG: --pod-cidr=""
May 29 02:52:29.629740 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626057    2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc"
May 29 02:52:29.629740 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626063    2559 flags.go:64] FLAG: --pod-manifest-path=""
May 29 02:52:29.629740 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626065    2559 flags.go:64] FLAG: --pod-max-pids="-1"
May 29 02:52:29.629740 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626069    2559 flags.go:64] FLAG: --pods-per-core="0"
May 29 02:52:29.629740 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626071    2559 flags.go:64] FLAG: --port="10250"
May 29 02:52:29.629740 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626075    2559 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 29 02:52:29.629740 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626077    2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0bbaf728441c77d56"
May 29 02:52:29.629740 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626081    2559 flags.go:64] FLAG: --qos-reserved=""
May 29 02:52:29.629740 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626084    2559 flags.go:64] FLAG: --read-only-port="10255"
May 29 02:52:29.630344 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626087    2559 flags.go:64] FLAG: --register-node="true"
May 29 02:52:29.630344 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626090    2559 flags.go:64] FLAG: --register-schedulable="true"
May 29 02:52:29.630344 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626092    2559 flags.go:64] FLAG: --register-with-taints=""
May 29 02:52:29.630344 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626099    2559 flags.go:64] FLAG: --registry-burst="10"
May 29 02:52:29.630344 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626102    2559 flags.go:64] FLAG: --registry-qps="5"
May 29 02:52:29.630344 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626107    2559 flags.go:64] FLAG: --reserved-cpus=""
May 29 02:52:29.630344 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626110    2559 flags.go:64] FLAG: --reserved-memory=""
May 29 02:52:29.630344 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626115    2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 29 02:52:29.630344 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626118    2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 29 02:52:29.630344 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626121    2559 flags.go:64] FLAG: --rotate-certificates="false"
May 29 02:52:29.630344 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626124    2559 flags.go:64] FLAG: --rotate-server-certificates="false"
May 29 02:52:29.630344 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626127    2559 flags.go:64] FLAG: --runonce="false"
May 29 02:52:29.630344 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626130    2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 29 02:52:29.630344 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626133    2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 29 02:52:29.630344 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626136    2559 flags.go:64] FLAG: --seccomp-default="false"
May 29 02:52:29.630344 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626139    2559 flags.go:64] FLAG: --serialize-image-pulls="true"
May 29 02:52:29.630344 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626142    2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 29 02:52:29.630344 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626145    2559 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 29 02:52:29.630344 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626148    2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 29 02:52:29.630344 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626151    2559 flags.go:64] FLAG: --storage-driver-password="root"
May 29 02:52:29.630344 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626154    2559 flags.go:64] FLAG: --storage-driver-secure="false"
May 29 02:52:29.630344 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626157    2559 flags.go:64] FLAG: --storage-driver-table="stats"
May 29 02:52:29.630344 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626160    2559 flags.go:64] FLAG: --storage-driver-user="root"
May 29 02:52:29.630344 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626163    2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 29 02:52:29.630344 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626166    2559 flags.go:64] FLAG: --sync-frequency="1m0s"
May 29 02:52:29.630344 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626169    2559 flags.go:64] FLAG: --system-cgroups=""
May 29 02:52:29.630982 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626171    2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 29 02:52:29.630982 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626176    2559 flags.go:64] FLAG: --system-reserved-cgroup=""
May 29 02:52:29.630982 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626179    2559 flags.go:64] FLAG: --tls-cert-file=""
May 29 02:52:29.630982 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626182    2559 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 29 02:52:29.630982 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626186    2559 flags.go:64] FLAG: --tls-min-version=""
May 29 02:52:29.630982 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626189    2559 flags.go:64] FLAG: --tls-private-key-file=""
May 29 02:52:29.630982 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626192    2559 flags.go:64] FLAG: --topology-manager-policy="none"
May 29 02:52:29.630982 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626194    2559 flags.go:64] FLAG: --topology-manager-policy-options=""
May 29 02:52:29.630982 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626197    2559 flags.go:64] FLAG: --topology-manager-scope="container"
May 29 02:52:29.630982 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626202    2559 flags.go:64] FLAG: --v="2"
May 29 02:52:29.630982 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626207    2559 flags.go:64] FLAG: --version="false"
May 29 02:52:29.630982 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626210    2559 flags.go:64] FLAG: --vmodule=""
May 29 02:52:29.630982 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626215    2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 29 02:52:29.630982 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.626219    2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 29 02:52:29.630982 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626308    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 29 02:52:29.630982 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626312    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 29 02:52:29.630982 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626315    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 29 02:52:29.630982 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626318    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 29 02:52:29.630982 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626321    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 29 02:52:29.630982 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626324    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 29 02:52:29.630982 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626328    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 29 02:52:29.630982 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626331    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 29 02:52:29.630982 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626334    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 29 02:52:29.631532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626338    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 29 02:52:29.631532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626342    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
May 29 02:52:29.631532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626345    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 29 02:52:29.631532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626348    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 29 02:52:29.631532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626351    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 29 02:52:29.631532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626354    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 29 02:52:29.631532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626356    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 29 02:52:29.631532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626359    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 29 02:52:29.631532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626361    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 29 02:52:29.631532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626364    2559 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 29 02:52:29.631532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626366    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 29 02:52:29.631532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626369    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 29 02:52:29.631532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626372    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 29 02:52:29.631532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626374    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 29 02:52:29.631532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626377    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 29 02:52:29.631532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626379    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 29 02:52:29.631532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626382    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 29 02:52:29.631532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626385    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 29 02:52:29.631532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626391    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 29 02:52:29.632058 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626394    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 29 02:52:29.632058 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626397    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 29 02:52:29.632058 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626400    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 29 02:52:29.632058 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626404    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 29 02:52:29.632058 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626407    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 29 02:52:29.632058 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626410    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 29 02:52:29.632058 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626412    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 29 02:52:29.632058 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626415    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 29 02:52:29.632058 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626418    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 29 02:52:29.632058 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626420    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 29 02:52:29.632058 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626422    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 29 02:52:29.632058 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626425    2559 feature_gate.go:328] unrecognized feature gate: Example
May 29 02:52:29.632058 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626428    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 29 02:52:29.632058 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626431    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 29 02:52:29.632058 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626434    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 29 02:52:29.632058 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626437    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 29 02:52:29.632058 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626439    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 29 02:52:29.632058 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626442    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 29 02:52:29.632058 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626445    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 29 02:52:29.632532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626447    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 29 02:52:29.632532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626450    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 29 02:52:29.632532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626452    2559 feature_gate.go:328] unrecognized feature gate: Example2
May 29 02:52:29.632532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626455    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 29 02:52:29.632532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626457    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 29 02:52:29.632532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626460    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 29 02:52:29.632532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626462    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 29 02:52:29.632532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626465    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 29 02:52:29.632532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626467    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 29 02:52:29.632532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626470    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 29 02:52:29.632532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626473    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 29 02:52:29.632532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626475    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 29 02:52:29.632532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626478    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 29 02:52:29.632532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626481    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 29 02:52:29.632532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626484    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 29 02:52:29.632532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626487    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 29 02:52:29.632532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626490    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 29 02:52:29.632532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626493    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 29 02:52:29.632532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626495    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 29 02:52:29.632532 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626498    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 29 02:52:29.633051 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626500    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 29 02:52:29.633051 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626503    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 29 02:52:29.633051 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626505    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 29 02:52:29.633051 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626508    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 29 02:52:29.633051 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626510    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 29 02:52:29.633051 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626513    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 29 02:52:29.633051 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626516    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 29 02:52:29.633051 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626523    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
May 29 02:52:29.633051 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626525    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 29 02:52:29.633051 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626528    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 29 02:52:29.633051 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626530    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 29 02:52:29.633051 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626533    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 29 02:52:29.633051 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626536    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 29 02:52:29.633051 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626538    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 29 02:52:29.633051 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626541    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 29 02:52:29.633051 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626543    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 29 02:52:29.633051 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626546    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 29 02:52:29.633051 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626549    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 29 02:52:29.633051 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.626551    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 29 02:52:29.633520 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.627503    2559 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 29 02:52:29.634345 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.634328    2559 server.go:530] "Kubelet version" kubeletVersion="v1.33.11"
May 29 02:52:29.634382 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.634345    2559 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 29 02:52:29.634415 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634397    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 29 02:52:29.634415 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634403    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 29 02:52:29.634415 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634406    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 29 02:52:29.634415 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634409    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 29 02:52:29.634415 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634412    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 29 02:52:29.634415 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634414    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 29 02:52:29.634415 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634418    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 29 02:52:29.634589 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634420    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 29 02:52:29.634589 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634424    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 29 02:52:29.634589 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634429    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 29 02:52:29.634589 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634432    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 29 02:52:29.634589 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634435    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 29 02:52:29.634589 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634438    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 29 02:52:29.634589 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634440    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 29 02:52:29.634589 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634443    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 29 02:52:29.634589 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634446    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 29 02:52:29.634589 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634449    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 29 02:52:29.634589 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634451    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 29 02:52:29.634589 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634454    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 29 02:52:29.634589 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634456    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
May 29 02:52:29.634589 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634459    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 29 02:52:29.634589 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634462    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 29 02:52:29.634589 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634464    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 29 02:52:29.634589 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634467    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 29 02:52:29.634589 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634469    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 29 02:52:29.634589 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634472    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 29 02:52:29.634589 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634474    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 29 02:52:29.635094 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634477    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 29 02:52:29.635094 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634479    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 29 02:52:29.635094 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634482    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 29 02:52:29.635094 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634484    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 29 02:52:29.635094 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634488    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 29 02:52:29.635094 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634490    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 29 02:52:29.635094 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634493    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 29 02:52:29.635094 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634495    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 29 02:52:29.635094 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634498    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 29 02:52:29.635094 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634500    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 29 02:52:29.635094 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634503    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 29 02:52:29.635094 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634505    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 29 02:52:29.635094 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634508    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 29 02:52:29.635094 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634511    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 29 02:52:29.635094 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634514    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 29 02:52:29.635094 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634516    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 29 02:52:29.635094 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634519    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 29 02:52:29.635094 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634522    2559 feature_gate.go:328] unrecognized feature gate: Example2
May 29 02:52:29.635094 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634524    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 29 02:52:29.635094 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634527    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 29 02:52:29.635632 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634529    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 29 02:52:29.635632 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634532    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 29 02:52:29.635632 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634534    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 29 02:52:29.635632 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634537    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 29 02:52:29.635632 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634539    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 29 02:52:29.635632 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634542    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
May 29 02:52:29.635632 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634544    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 29 02:52:29.635632 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634547    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 29 02:52:29.635632 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634551    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 29 02:52:29.635632 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634554    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 29 02:52:29.635632 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634558    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 29 02:52:29.635632 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634560    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 29 02:52:29.635632 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634564    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 29 02:52:29.635632 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634566    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 29 02:52:29.635632 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634569    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 29 02:52:29.635632 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634572    2559 feature_gate.go:328] unrecognized feature gate: Example
May 29 02:52:29.635632 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634575    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 29 02:52:29.635632 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634578    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 29 02:52:29.635632 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634581    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 29 02:52:29.635632 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634584    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 29 02:52:29.636133 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634586    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 29 02:52:29.636133 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634588    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 29 02:52:29.636133 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634591    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 29 02:52:29.636133 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634594    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 29 02:52:29.636133 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634597    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 29 02:52:29.636133 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634600    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 29 02:52:29.636133 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634602    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 29 02:52:29.636133 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634605    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 29 02:52:29.636133 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634608    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 29 02:52:29.636133 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634610    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 29 02:52:29.636133 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634613    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 29 02:52:29.636133 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634615    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 29 02:52:29.636133 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634618    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 29 02:52:29.636133 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634620    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 29 02:52:29.636133 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634622    2559 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 29 02:52:29.636133 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634625    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 29 02:52:29.636133 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634627    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 29 02:52:29.636133 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634630    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 29 02:52:29.636133 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634633    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 29 02:52:29.636607 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.634638    2559 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 29 02:52:29.636607 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634732    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 29 02:52:29.636607 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634736    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 29 02:52:29.636607 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634739    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 29 02:52:29.636607 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634742    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 29 02:52:29.636607 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634745    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 29 02:52:29.636607 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634747    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 29 02:52:29.636607 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634750    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 29 02:52:29.636607 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634753    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 29 02:52:29.636607 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634756    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 29 02:52:29.636607 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634758    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 29 02:52:29.636607 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634761    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 29 02:52:29.636607 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634764    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 29 02:52:29.636607 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634766    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 29 02:52:29.636607 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634769    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 29 02:52:29.637014 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634771    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 29 02:52:29.637014 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634774    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 29 02:52:29.637014 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634776    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 29 02:52:29.637014 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634779    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 29 02:52:29.637014 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634782    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 29 02:52:29.637014 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634785    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 29 02:52:29.637014 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634787    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 29 02:52:29.637014 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634790    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 29 02:52:29.637014 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634792    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 29 02:52:29.637014 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634795    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 29 02:52:29.637014 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634797    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 29 02:52:29.637014 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634800    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 29 02:52:29.637014 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634803    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 29 02:52:29.637014 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634805    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 29 02:52:29.637014 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634808    2559 feature_gate.go:328] unrecognized feature gate: Example2
May 29 02:52:29.637014 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634810    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 29 02:52:29.637014 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634813    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 29 02:52:29.637014 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634815    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
May 29 02:52:29.637014 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634818    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 29 02:52:29.637014 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634820    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 29 02:52:29.637014 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634823    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 29 02:52:29.637522 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634825    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 29 02:52:29.637522 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634828    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 29 02:52:29.637522 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634831    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 29 02:52:29.637522 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634834    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 29 02:52:29.637522 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634837    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 29 02:52:29.637522 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634840    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 29 02:52:29.637522 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634842    2559 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 29 02:52:29.637522 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634845    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 29 02:52:29.637522 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634848    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 29 02:52:29.637522 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634850    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 29 02:52:29.637522 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634853    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 29 02:52:29.637522 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634855    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 29 02:52:29.637522 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634858    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 29 02:52:29.637522 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634860    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 29 02:52:29.637522 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634863    2559 feature_gate.go:328] unrecognized feature gate: Example
May 29 02:52:29.637522 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634866    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 29 02:52:29.637522 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634869    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 29 02:52:29.637522 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634871    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 29 02:52:29.637522 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634873    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 29 02:52:29.638026 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634876    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 29 02:52:29.638026 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634878    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 29 02:52:29.638026 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634882    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 29 02:52:29.638026 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634886    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 29 02:52:29.638026 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634889    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 29 02:52:29.638026 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634892    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 29 02:52:29.638026 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634895    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 29 02:52:29.638026 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634898    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 29 02:52:29.638026 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634901    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 29 02:52:29.638026 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634903    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 29 02:52:29.638026 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634906    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 29 02:52:29.638026 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634908    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 29 02:52:29.638026 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634911    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 29 02:52:29.638026 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634913    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 29 02:52:29.638026 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634916    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 29 02:52:29.638026 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634918    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 29 02:52:29.638026 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634921    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 29 02:52:29.638026 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634923    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 29 02:52:29.638026 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634925    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 29 02:52:29.638475 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634928    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 29 02:52:29.638475 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634930    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 29 02:52:29.638475 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634933    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 29 02:52:29.638475 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634936    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 29 02:52:29.638475 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634938    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 29 02:52:29.638475 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634941    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 29 02:52:29.638475 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634943    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 29 02:52:29.638475 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634946    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 29 02:52:29.638475 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634949    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 29 02:52:29.638475 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634951    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
May 29 02:52:29.638475 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634954    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 29 02:52:29.638475 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634956    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 29 02:52:29.638475 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:29.634959    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 29 02:52:29.638475 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.634963    2559 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 29 02:52:29.638475 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.635858    2559 server.go:962] "Client rotation is on, will bootstrap in background"
May 29 02:52:29.638833 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.638326    2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 29 02:52:29.639439 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.639428    2559 server.go:1019] "Starting client certificate rotation"
May 29 02:52:29.639543 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.639528    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 29 02:52:29.639571 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.639557    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 29 02:52:29.667987 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.667936    2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 29 02:52:29.673112 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.673095    2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 29 02:52:29.687287 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.687271    2559 log.go:25] "Validated CRI v1 runtime API"
May 29 02:52:29.693445 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.693431    2559 log.go:25] "Validated CRI v1 image API"
May 29 02:52:29.694777 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.694761    2559 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 29 02:52:29.699176 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.699158    2559 fs.go:135] Filesystem UUIDs: map[72a47dbf-2e0c-427e-a7de-29bcc77ab515:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 b75fc135-907e-4580-9d7a-3da6f2f453d4:/dev/nvme0n1p4]
May 29 02:52:29.699251 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.699175    2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 29 02:52:29.703313 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.703206    2559 manager.go:217] Machine: {Timestamp:2026-05-29 02:52:29.702700345 +0000 UTC m=+0.458015400 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098869 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2de8490783b4c85f414add9f6a2173 SystemUUID:ec2de849-0783-b4c8-5f41-4add9f6a2173 BootID:d34ca446-1747-41ec-a91a-256154710a72 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:dd:9e:6c:24:19 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:dd:9e:6c:24:19 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:10:ae:7a:fa:a2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 29 02:52:29.703680 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.703666    2559 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 29 02:52:29.704888 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.704877    2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 29 02:52:29.704978 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.704956    2559 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 29 02:52:29.707054 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.707027    2559 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 29 02:52:29.707183 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.707057    2559 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-122.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 29 02:52:29.707228 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.707192    2559 topology_manager.go:138] "Creating topology manager with none policy"
May 29 02:52:29.707228 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.707201    2559 container_manager_linux.go:306] "Creating device plugin manager"
May 29 02:52:29.707228 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.707208    2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 29 02:52:29.708103 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.708092    2559 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 29 02:52:29.710805 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.710795    2559 state_mem.go:36] "Initialized new in-memory state store"
May 29 02:52:29.710917 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.710908    2559 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 29 02:52:29.713703 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.713694    2559 kubelet.go:491] "Attempting to sync node with API server"
May 29 02:52:29.713743 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.713712    2559 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 29 02:52:29.713743 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.713724    2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 29 02:52:29.713743 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.713734    2559 kubelet.go:397] "Adding apiserver pod source"
May 29 02:52:29.713743 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.713741    2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 29 02:52:29.714937 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.714926    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 29 02:52:29.714996 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.714944    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 29 02:52:29.718295 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.718281    2559 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 29 02:52:29.721012 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.720999    2559 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 29 02:52:29.721408 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.721381    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rxqww"
May 29 02:52:29.722005 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.721993    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 29 02:52:29.722047 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.722010    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 29 02:52:29.722047 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.722016    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 29 02:52:29.722047 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.722022    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 29 02:52:29.722047 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.722030    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 29 02:52:29.722047 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.722035    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 29 02:52:29.722047 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.722041    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 29 02:52:29.722047 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.722046    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 29 02:52:29.722235 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.722053    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 29 02:52:29.722235 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.722058    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 29 02:52:29.722235 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.722072    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 29 02:52:29.722469 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:29.722450    2559 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 29 02:52:29.722518 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.722473    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 29 02:52:29.722548 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:29.722519    2559 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-122.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 29 02:52:29.723526 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.723516    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 29 02:52:29.723569 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.723527    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 29 02:52:29.725745 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.725732    2559 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-122.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 29 02:52:29.726636 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.726623    2559 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 29 02:52:29.726671 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.726660    2559 server.go:1295] "Started kubelet"
May 29 02:52:29.726785 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.726758    2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 29 02:52:29.726846 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.726755    2559 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 29 02:52:29.726846 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.726820    2559 server_v1.go:47] "podresources" method="list" useActivePods=true
May 29 02:52:29.727397 ip-10-0-137-122 systemd[1]: Started Kubernetes Kubelet.
May 29 02:52:29.728776 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.728760    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rxqww"
May 29 02:52:29.733534 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.733253    2559 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 29 02:52:29.733534 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.733323    2559 server.go:317] "Adding debug handlers to kubelet server"
May 29 02:52:29.740056 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:29.740036    2559 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 29 02:52:29.740833 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.740822    2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 29 02:52:29.740894 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.740835    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 29 02:52:29.740944 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.740893    2559 volume_manager.go:295] "The desired_state_of_world populator starts"
May 29 02:52:29.740944 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.740905    2559 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 29 02:52:29.741060 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.741001    2559 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 29 02:52:29.741060 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.741059    2559 reconstruct.go:97] "Volume reconstruction finished"
May 29 02:52:29.741161 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.741068    2559 reconciler.go:26] "Reconciler: start to sync state"
May 29 02:52:29.741474 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.741458    2559 factory.go:55] Registering systemd factory
May 29 02:52:29.741566 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.741543    2559 factory.go:223] Registration of the systemd container factory successfully
May 29 02:52:29.741769 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.741752    2559 factory.go:153] Registering CRI-O factory
May 29 02:52:29.741769 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.741769    2559 factory.go:223] Registration of the crio container factory successfully
May 29 02:52:29.741907 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:29.741777    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-122.ec2.internal\" not found"
May 29 02:52:29.741907 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.741815    2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 29 02:52:29.741907 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.741834    2559 factory.go:103] Registering Raw factory
May 29 02:52:29.741907 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.741844    2559 manager.go:1196] Started watching for new ooms in manager
May 29 02:52:29.742263 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.742208    2559 manager.go:319] Starting recovery of all containers
May 29 02:52:29.748813 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.748790    2559 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 02:52:29.752258 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.752158    2559 manager.go:324] Recovery completed
May 29 02:52:29.753405 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:29.753382    2559 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
May 29 02:52:29.753517 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:29.753503    2559 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-137-122.ec2.internal\" not found" node="ip-10-0-137-122.ec2.internal"
May 29 02:52:29.756259 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.756247    2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 02:52:29.758013 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.758000    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-122.ec2.internal" event="NodeHasSufficientMemory"
May 29 02:52:29.758078 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.758023    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-122.ec2.internal" event="NodeHasNoDiskPressure"
May 29 02:52:29.758078 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.758035    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-122.ec2.internal" event="NodeHasSufficientPID"
May 29 02:52:29.758446 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.758435    2559 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 29 02:52:29.758499 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.758445    2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 29 02:52:29.758499 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.758461    2559 state_mem.go:36] "Initialized new in-memory state store"
May 29 02:52:29.762230 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.762217    2559 policy_none.go:49] "None policy: Start"
May 29 02:52:29.762280 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.762233    2559 memory_manager.go:186] "Starting memorymanager" policy="None"
May 29 02:52:29.762280 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.762243    2559 state_mem.go:35] "Initializing new in-memory state store"
May 29 02:52:29.796506 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.796492    2559 manager.go:341] "Starting Device Plugin manager"
May 29 02:52:29.824202 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:29.796534    2559 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 29 02:52:29.824202 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.796546    2559 server.go:85] "Starting device plugin registration server"
May 29 02:52:29.824202 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.796756    2559 eviction_manager.go:189] "Eviction manager: starting control loop"
May 29 02:52:29.824202 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.796768    2559 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 29 02:52:29.824202 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.796859    2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 29 02:52:29.824202 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.796926    2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 29 02:52:29.824202 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.796935    2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 29 02:52:29.824202 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:29.797464    2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 29 02:52:29.824202 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:29.797499    2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-122.ec2.internal\" not found"
May 29 02:52:29.842938 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.842913    2559 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 29 02:52:29.844008 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.843987    2559 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 29 02:52:29.844078 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.844011    2559 status_manager.go:230] "Starting to sync pod status with apiserver"
May 29 02:52:29.844078 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.844027    2559 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 29 02:52:29.844078 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.844033    2559 kubelet.go:2451] "Starting kubelet main sync loop"
May 29 02:52:29.844078 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:29.844067    2559 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 29 02:52:29.846360 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.846342    2559 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 02:52:29.897316 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.897293    2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 02:52:29.898213 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.898199    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-122.ec2.internal" event="NodeHasSufficientMemory"
May 29 02:52:29.898278 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.898226    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-122.ec2.internal" event="NodeHasNoDiskPressure"
May 29 02:52:29.898278 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.898236    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-122.ec2.internal" event="NodeHasSufficientPID"
May 29 02:52:29.898278 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.898256    2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-122.ec2.internal"
May 29 02:52:29.905239 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.905225    2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-122.ec2.internal"
May 29 02:52:29.905314 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:29.905244    2559 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-122.ec2.internal\": node \"ip-10-0-137-122.ec2.internal\" not found"
May 29 02:52:29.924772 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:29.924723    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-122.ec2.internal\" not found"
May 29 02:52:29.944578 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.944552    2559 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-137-122.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-122.ec2.internal"]
May 29 02:52:29.944645 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.944624    2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 02:52:29.945402 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.945386    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-122.ec2.internal" event="NodeHasSufficientMemory"
May 29 02:52:29.945479 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.945417    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-122.ec2.internal" event="NodeHasNoDiskPressure"
May 29 02:52:29.945479 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.945428    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-122.ec2.internal" event="NodeHasSufficientPID"
May 29 02:52:29.946783 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.946772    2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 02:52:29.946916 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.946903    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-122.ec2.internal"
May 29 02:52:29.946986 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.946930    2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 02:52:29.947816 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.947803    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-122.ec2.internal" event="NodeHasSufficientMemory"
May 29 02:52:29.947816 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.947813    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-122.ec2.internal" event="NodeHasSufficientMemory"
May 29 02:52:29.947948 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.947828    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-122.ec2.internal" event="NodeHasNoDiskPressure"
May 29 02:52:29.947948 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.947835    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-122.ec2.internal" event="NodeHasNoDiskPressure"
May 29 02:52:29.947948 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.947843    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-122.ec2.internal" event="NodeHasSufficientPID"
May 29 02:52:29.947948 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.947847    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-122.ec2.internal" event="NodeHasSufficientPID"
May 29 02:52:29.949176 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.949161    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-122.ec2.internal"
May 29 02:52:29.949256 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.949189    2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 02:52:29.949984 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.949956    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-122.ec2.internal" event="NodeHasSufficientMemory"
May 29 02:52:29.950039 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.949998    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-122.ec2.internal" event="NodeHasNoDiskPressure"
May 29 02:52:29.950039 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:29.950008    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-122.ec2.internal" event="NodeHasSufficientPID"
May 29 02:52:29.969535 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:29.969519    2559 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-122.ec2.internal\" not found" node="ip-10-0-137-122.ec2.internal"
May 29 02:52:29.973872 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:29.973857    2559 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-122.ec2.internal\" not found" node="ip-10-0-137-122.ec2.internal"
May 29 02:52:30.025003 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:30.024984    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-122.ec2.internal\" not found"
May 29 02:52:30.041875 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.041857    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fcf3ae000999ff406cf9a785430af920-config\") pod \"kube-apiserver-proxy-ip-10-0-137-122.ec2.internal\" (UID: \"fcf3ae000999ff406cf9a785430af920\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-122.ec2.internal"
May 29 02:52:30.125317 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:30.125296    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-122.ec2.internal\" not found"
May 29 02:52:30.142594 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.142573    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fcf3ae000999ff406cf9a785430af920-config\") pod \"kube-apiserver-proxy-ip-10-0-137-122.ec2.internal\" (UID: \"fcf3ae000999ff406cf9a785430af920\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-122.ec2.internal"
May 29 02:52:30.142651 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.142602    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/10985281f2f37314cf306f08a4b9cf5b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-122.ec2.internal\" (UID: \"10985281f2f37314cf306f08a4b9cf5b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-122.ec2.internal"
May 29 02:52:30.142690 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.142655    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fcf3ae000999ff406cf9a785430af920-config\") pod \"kube-apiserver-proxy-ip-10-0-137-122.ec2.internal\" (UID: \"fcf3ae000999ff406cf9a785430af920\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-122.ec2.internal"
May 29 02:52:30.142731 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.142696    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10985281f2f37314cf306f08a4b9cf5b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-122.ec2.internal\" (UID: \"10985281f2f37314cf306f08a4b9cf5b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-122.ec2.internal"
May 29 02:52:30.226046 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:30.225987    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-122.ec2.internal\" not found"
May 29 02:52:30.243255 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.243226    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/10985281f2f37314cf306f08a4b9cf5b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-122.ec2.internal\" (UID: \"10985281f2f37314cf306f08a4b9cf5b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-122.ec2.internal"
May 29 02:52:30.243317 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.243263    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10985281f2f37314cf306f08a4b9cf5b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-122.ec2.internal\" (UID: \"10985281f2f37314cf306f08a4b9cf5b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-122.ec2.internal"
May 29 02:52:30.243317 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.243289    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10985281f2f37314cf306f08a4b9cf5b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-122.ec2.internal\" (UID: \"10985281f2f37314cf306f08a4b9cf5b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-122.ec2.internal"
May 29 02:52:30.243317 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.243306    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/10985281f2f37314cf306f08a4b9cf5b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-122.ec2.internal\" (UID: \"10985281f2f37314cf306f08a4b9cf5b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-122.ec2.internal"
May 29 02:52:30.272380 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.272359    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-122.ec2.internal"
May 29 02:52:30.277021 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.277005    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-122.ec2.internal"
May 29 02:52:30.326915 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:30.326891    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-122.ec2.internal\" not found"
May 29 02:52:30.427364 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:30.427336    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-122.ec2.internal\" not found"
May 29 02:52:30.527473 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:30.527414    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-122.ec2.internal\" not found"
May 29 02:52:30.527601 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.527586    2559 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 02:52:30.541615 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.541597    2559 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-122.ec2.internal"
May 29 02:52:30.551284 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.551266    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 29 02:52:30.552910 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.552898    2559 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-122.ec2.internal"
May 29 02:52:30.563520 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.563492    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 29 02:52:30.639552 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.639533    2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 29 02:52:30.640010 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.639633    2559 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 29 02:52:30.640010 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.639675    2559 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 29 02:52:30.640010 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.639681    2559 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 29 02:52:30.714028 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.714012    2559 apiserver.go:52] "Watching apiserver"
May 29 02:52:30.730030 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.730010    2559 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 29 02:52:30.732148 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.732126    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-137-122.ec2.internal","openshift-dns/node-resolver-wz44t","openshift-image-registry/node-ca-92nfp","openshift-multus/multus-7fhjp","openshift-multus/multus-additional-cni-plugins-22clf","openshift-network-operator/iptables-alerter-zb5lm","openshift-ovn-kubernetes/ovnkube-node-psj9n","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2","openshift-cluster-node-tuning-operator/tuned-c8wpt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-122.ec2.internal","openshift-multus/network-metrics-daemon-r4cd8","openshift-network-diagnostics/network-check-target-fnzpl","kube-system/konnectivity-agent-h2vvt"]
May 29 02:52:30.736101 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.736080    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-28 02:47:29 +0000 UTC" deadline="2028-02-09 04:18:55.732435796 +0000 UTC"
May 29 02:52:30.736161 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.736102    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14905h26m24.996336914s"
May 29 02:52:30.736448 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.736424    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:52:30.736562 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:30.736519    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4cd8" podUID="757d1101-4ef1-451f-b984-012545775d40"
May 29 02:52:30.738617 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.738592    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h2vvt"
May 29 02:52:30.738841 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.738757    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.740500 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.740482    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 29 02:52:30.740748 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.740730    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.740808 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.740780    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-tr5lz\""
May 29 02:52:30.741360 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.741339    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 29 02:52:30.742505 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.741735    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 29 02:52:30.742505 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.741904    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 29 02:52:30.742505 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.742392    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4g58k\""
May 29 02:52:30.743010 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.742989    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 29 02:52:30.743354 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.743338    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 29 02:52:30.743416 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.743384    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 29 02:52:30.743500 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.743336    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 29 02:52:30.743587 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.743574    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 29 02:52:30.743704 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.743690    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 29 02:52:30.743808 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.743788    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 29 02:52:30.743947 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.743929    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 29 02:52:30.744018 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.743984    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 29 02:52:30.744876 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.744854    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5bvjx\""
May 29 02:52:30.744997 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.744911    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-host-run-netns\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.744997 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.744946    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-etc-openvswitch\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.744997 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.744991    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-multus-conf-dir\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.745190 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745066    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/532da6c4-ac98-4477-88e3-021b3a046918-multus-daemon-config\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.745190 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745093    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-host-kubelet\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.745190 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745112    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-run-systemd\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.745190 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745136    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-node-log\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.745190 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745154    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-host-cni-bin\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.745190 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745169    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cccb57a5-0894-4319-9eeb-2459f94a7a17-system-cni-dir\") pod \"multus-additional-cni-plugins-22clf\" (UID: \"cccb57a5-0894-4319-9eeb-2459f94a7a17\") " pod="openshift-multus/multus-additional-cni-plugins-22clf"
May 29 02:52:30.745190 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745187    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cccb57a5-0894-4319-9eeb-2459f94a7a17-tuning-conf-dir\") pod \"multus-additional-cni-plugins-22clf\" (UID: \"cccb57a5-0894-4319-9eeb-2459f94a7a17\") " pod="openshift-multus/multus-additional-cni-plugins-22clf"
May 29 02:52:30.745525 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745211    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-log-socket\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.745525 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745237    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d9a08de9-bea6-4561-8eae-62421b7584a9-ovn-node-metrics-cert\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.745525 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745262    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d9a08de9-bea6-4561-8eae-62421b7584a9-ovnkube-script-lib\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.745525 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745285    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22dg5\" (UniqueName: \"kubernetes.io/projected/d9a08de9-bea6-4561-8eae-62421b7584a9-kube-api-access-22dg5\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.745525 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745306    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/532da6c4-ac98-4477-88e3-021b3a046918-cni-binary-copy\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.745525 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745321    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5nlf\" (UniqueName: \"kubernetes.io/projected/532da6c4-ac98-4477-88e3-021b3a046918-kube-api-access-k5nlf\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.745525 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745337    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cccb57a5-0894-4319-9eeb-2459f94a7a17-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-22clf\" (UID: \"cccb57a5-0894-4319-9eeb-2459f94a7a17\") " pod="openshift-multus/multus-additional-cni-plugins-22clf"
May 29 02:52:30.745525 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745360    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2vfm\" (UniqueName: \"kubernetes.io/projected/757d1101-4ef1-451f-b984-012545775d40-kube-api-access-j2vfm\") pod \"network-metrics-daemon-r4cd8\" (UID: \"757d1101-4ef1-451f-b984-012545775d40\") " pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:52:30.745525 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745378    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-run-openvswitch\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.745525 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745397    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-host-run-ovn-kubernetes\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.745525 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745426    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-system-cni-dir\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.745525 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745451    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-multus-cni-dir\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.745525 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745465    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-host-run-netns\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.745525 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745480    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-host-run-multus-certs\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.745525 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745498    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-etc-kubernetes\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.745525 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745522    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-host-slash\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.746381 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745543    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-host-cni-netd\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.746381 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745560    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-cnibin\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.746381 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745574    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cccb57a5-0894-4319-9eeb-2459f94a7a17-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-22clf\" (UID: \"cccb57a5-0894-4319-9eeb-2459f94a7a17\") " pod="openshift-multus/multus-additional-cni-plugins-22clf"
May 29 02:52:30.746381 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745590    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs\") pod \"network-metrics-daemon-r4cd8\" (UID: \"757d1101-4ef1-451f-b984-012545775d40\") " pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:52:30.746381 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745604    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/deaa9f6f-56b4-4875-85fc-a2396ad84029-agent-certs\") pod \"konnectivity-agent-h2vvt\" (UID: \"deaa9f6f-56b4-4875-85fc-a2396ad84029\") " pod="kube-system/konnectivity-agent-h2vvt"
May 29 02:52:30.746381 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745617    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d9a08de9-bea6-4561-8eae-62421b7584a9-ovnkube-config\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.746381 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745629    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d9a08de9-bea6-4561-8eae-62421b7584a9-env-overrides\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.746381 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745642    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-var-lib-openvswitch\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.746381 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745656    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-run-ovn\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.746381 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745672    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.746381 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745706    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-multus-socket-dir-parent\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.746381 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745735    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-host-var-lib-cni-bin\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.746381 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745757    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-host-var-lib-kubelet\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.746381 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745771    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-systemd-units\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.746381 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745787    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-os-release\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.746381 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745809    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-host-run-k8s-cni-cncf-io\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.747169 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745825    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-host-var-lib-cni-multus\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.747169 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745838    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-hostroot\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.747169 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745853    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cccb57a5-0894-4319-9eeb-2459f94a7a17-os-release\") pod \"multus-additional-cni-plugins-22clf\" (UID: \"cccb57a5-0894-4319-9eeb-2459f94a7a17\") " pod="openshift-multus/multus-additional-cni-plugins-22clf"
May 29 02:52:30.747169 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745873    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/deaa9f6f-56b4-4875-85fc-a2396ad84029-konnectivity-ca\") pod \"konnectivity-agent-h2vvt\" (UID: \"deaa9f6f-56b4-4875-85fc-a2396ad84029\") " pod="kube-system/konnectivity-agent-h2vvt"
May 29 02:52:30.747169 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745887    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cccb57a5-0894-4319-9eeb-2459f94a7a17-cnibin\") pod \"multus-additional-cni-plugins-22clf\" (UID: \"cccb57a5-0894-4319-9eeb-2459f94a7a17\") " pod="openshift-multus/multus-additional-cni-plugins-22clf"
May 29 02:52:30.747169 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745901    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cccb57a5-0894-4319-9eeb-2459f94a7a17-cni-binary-copy\") pod \"multus-additional-cni-plugins-22clf\" (UID: \"cccb57a5-0894-4319-9eeb-2459f94a7a17\") " pod="openshift-multus/multus-additional-cni-plugins-22clf"
May 29 02:52:30.747169 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.745918    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wckc4\" (UniqueName: \"kubernetes.io/projected/cccb57a5-0894-4319-9eeb-2459f94a7a17-kube-api-access-wckc4\") pod \"multus-additional-cni-plugins-22clf\" (UID: \"cccb57a5-0894-4319-9eeb-2459f94a7a17\") " pod="openshift-multus/multus-additional-cni-plugins-22clf"
May 29 02:52:30.747169 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.746378    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-22clf"
May 29 02:52:30.747169 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.746442    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zb5lm"
May 29 02:52:30.747889 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.747874    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 29 02:52:30.748066 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.747904    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 29 02:52:30.748066 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.747989    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-c6kpz\""
May 29 02:52:30.748304 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.748290    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 29 02:52:30.748375 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.748313    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 29 02:52:30.748375 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.748328    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wg7m9\""
May 29 02:52:30.748473 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.748387    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 29 02:52:30.748667 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.748651    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2"
May 29 02:52:30.751140 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.750372    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jnhw7\""
May 29 02:52:30.751140 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.750381    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 29 02:52:30.751140 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.750621    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 29 02:52:30.751140 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.750767    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 29 02:52:30.751140 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.750933    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.752086 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.752072    2559 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 29 02:52:30.752349 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.752331    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tw5g8\""
May 29 02:52:30.752470 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.752456    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 29 02:52:30.752539 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.752528    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 29 02:52:30.753186 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.753172    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-92nfp"
May 29 02:52:30.754453 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.754439    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 29 02:52:30.754527 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.754511    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bwb9l\""
May 29 02:52:30.754679 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.754666    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 29 02:52:30.754714 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.754668    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 29 02:52:30.755251 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.755239    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:52:30.755311 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:30.755295    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnzpl" podUID="902df08f-86be-4b22-8929-a7087c7b2c9a"
May 29 02:52:30.757304 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.757291    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wz44t"
May 29 02:52:30.758620 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.758606    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 29 02:52:30.758755 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.758741    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cqpcw\""
May 29 02:52:30.758793 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.758786    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 29 02:52:30.783626 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.783610    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8ngnq"
May 29 02:52:30.791417 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.791401    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8ngnq"
May 29 02:52:30.841700 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.841683    2559 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 29 02:52:30.846651 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.846633    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-host-kubelet\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.846733 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.846659    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-run-systemd\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.846733 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.846677    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-host-cni-bin\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.846733 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.846695    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-etc-sysconfig\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.846733 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.846710    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/32a9fc68-65c5-42d9-9121-6fa454c910f4-iptables-alerter-script\") pod \"iptables-alerter-zb5lm\" (UID: \"32a9fc68-65c5-42d9-9121-6fa454c910f4\") " pod="openshift-network-operator/iptables-alerter-zb5lm"
May 29 02:52:30.846858 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.846740    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-host-kubelet\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.846858 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.846752    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-host-cni-bin\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.846858 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.846743    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-run-systemd\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.846858 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.846800    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/32a9fc68-65c5-42d9-9121-6fa454c910f4-host-slash\") pod \"iptables-alerter-zb5lm\" (UID: \"32a9fc68-65c5-42d9-9121-6fa454c910f4\") " pod="openshift-network-operator/iptables-alerter-zb5lm"
May 29 02:52:30.846858 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.846820    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h7rh2\" (UID: \"adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2"
May 29 02:52:30.846858 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.846836    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0-etc-selinux\") pod \"aws-ebs-csi-driver-node-h7rh2\" (UID: \"adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2"
May 29 02:52:30.846858 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.846853    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-log-socket\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.847075 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.846887    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d9a08de9-bea6-4561-8eae-62421b7584a9-ovn-node-metrics-cert\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.847075 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.846912    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-log-socket\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.847075 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.846948    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/532da6c4-ac98-4477-88e3-021b3a046918-cni-binary-copy\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.847075 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.846982    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cccb57a5-0894-4319-9eeb-2459f94a7a17-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-22clf\" (UID: \"cccb57a5-0894-4319-9eeb-2459f94a7a17\") " pod="openshift-multus/multus-additional-cni-plugins-22clf"
May 29 02:52:30.847075 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847008    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-var-lib-kubelet\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.847075 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847023    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-tmp\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.847075 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847038    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-multus-cni-dir\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.847075 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847060    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-host-run-netns\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.847398 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847101    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-host-run-multus-certs\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.847398 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847126    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-etc-kubernetes\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.847398 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847140    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-multus-cni-dir\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.847398 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847149    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs\") pod \"network-metrics-daemon-r4cd8\" (UID: \"757d1101-4ef1-451f-b984-012545775d40\") " pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:52:30.847398 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847152    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-host-run-netns\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.847398 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847173    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-etc-sysctl-conf\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.847398 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847187    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-host-run-multus-certs\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.847398 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847199    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw95k\" (UniqueName: \"kubernetes.io/projected/235d8ad4-77e9-477a-b8d4-4edc46151e4c-kube-api-access-vw95k\") pod \"node-ca-92nfp\" (UID: \"235d8ad4-77e9-477a-b8d4-4edc46151e4c\") " pod="openshift-image-registry/node-ca-92nfp"
May 29 02:52:30.847398 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847200    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-etc-kubernetes\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.847398 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847225    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-host-cni-netd\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.847398 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847251    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/deaa9f6f-56b4-4875-85fc-a2396ad84029-agent-certs\") pod \"konnectivity-agent-h2vvt\" (UID: \"deaa9f6f-56b4-4875-85fc-a2396ad84029\") " pod="kube-system/konnectivity-agent-h2vvt"
May 29 02:52:30.847398 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847266    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-host-cni-netd\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.847398 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:30.847274    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:52:30.847398 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847277    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-etc-kubernetes\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.847398 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847362    2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 29 02:52:30.847398 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:30.847370    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs podName:757d1101-4ef1-451f-b984-012545775d40 nodeName:}" failed. No retries permitted until 2026-05-29 02:52:31.347323158 +0000 UTC m=+2.102638425 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs") pod "network-metrics-daemon-r4cd8" (UID: "757d1101-4ef1-451f-b984-012545775d40") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:52:30.847398 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847401    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-etc-tuned\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.848160 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847426    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0-sys-fs\") pod \"aws-ebs-csi-driver-node-h7rh2\" (UID: \"adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2"
May 29 02:52:30.848160 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847451    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d9a08de9-bea6-4561-8eae-62421b7584a9-ovnkube-config\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.848160 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847472    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-var-lib-openvswitch\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.848160 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847483    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/532da6c4-ac98-4477-88e3-021b3a046918-cni-binary-copy\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.848160 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847499    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-run-ovn\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.848160 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847528    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.848160 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847552    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-host-var-lib-cni-bin\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.848160 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847558    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-var-lib-openvswitch\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.848160 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847551    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cccb57a5-0894-4319-9eeb-2459f94a7a17-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-22clf\" (UID: \"cccb57a5-0894-4319-9eeb-2459f94a7a17\") " pod="openshift-multus/multus-additional-cni-plugins-22clf"
May 29 02:52:30.848160 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847598    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-host\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.848160 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847635    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-host-var-lib-cni-bin\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.848160 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847646    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.848160 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847634    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-run-ovn\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.848160 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847676    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrsj8\" (UniqueName: \"kubernetes.io/projected/adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0-kube-api-access-hrsj8\") pod \"aws-ebs-csi-driver-node-h7rh2\" (UID: \"adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2"
May 29 02:52:30.848160 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847717    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/846260de-5ed8-4c80-ad40-8480a09eddd2-hosts-file\") pod \"node-resolver-wz44t\" (UID: \"846260de-5ed8-4c80-ad40-8480a09eddd2\") " pod="openshift-dns/node-resolver-wz44t"
May 29 02:52:30.848160 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847752    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-systemd-units\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.848160 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847777    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-os-release\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.848917 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847801    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-host-run-k8s-cni-cncf-io\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.848917 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847809    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-systemd-units\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.848917 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847826    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/deaa9f6f-56b4-4875-85fc-a2396ad84029-konnectivity-ca\") pod \"konnectivity-agent-h2vvt\" (UID: \"deaa9f6f-56b4-4875-85fc-a2396ad84029\") " pod="kube-system/konnectivity-agent-h2vvt"
May 29 02:52:30.848917 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847853    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zw6qf\" (UniqueName: \"kubernetes.io/projected/902df08f-86be-4b22-8929-a7087c7b2c9a-kube-api-access-zw6qf\") pod \"network-check-target-fnzpl\" (UID: \"902df08f-86be-4b22-8929-a7087c7b2c9a\") " pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:52:30.848917 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847871    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-os-release\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.848917 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847876    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-run\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.848917 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847855    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-host-run-k8s-cni-cncf-io\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.848917 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847900    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdx6c\" (UniqueName: \"kubernetes.io/projected/846260de-5ed8-4c80-ad40-8480a09eddd2-kube-api-access-xdx6c\") pod \"node-resolver-wz44t\" (UID: \"846260de-5ed8-4c80-ad40-8480a09eddd2\") " pod="openshift-dns/node-resolver-wz44t"
May 29 02:52:30.848917 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847925    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-etc-modprobe-d\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.848917 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847949    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/235d8ad4-77e9-477a-b8d4-4edc46151e4c-host\") pod \"node-ca-92nfp\" (UID: \"235d8ad4-77e9-477a-b8d4-4edc46151e4c\") " pod="openshift-image-registry/node-ca-92nfp"
May 29 02:52:30.848917 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847957    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d9a08de9-bea6-4561-8eae-62421b7584a9-ovnkube-config\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.848917 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.847987    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-etc-sysctl-d\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.848917 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848018    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-node-log\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.848917 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848065    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-node-log\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.848917 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848092    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cccb57a5-0894-4319-9eeb-2459f94a7a17-system-cni-dir\") pod \"multus-additional-cni-plugins-22clf\" (UID: \"cccb57a5-0894-4319-9eeb-2459f94a7a17\") " pod="openshift-multus/multus-additional-cni-plugins-22clf"
May 29 02:52:30.848917 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848131    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cccb57a5-0894-4319-9eeb-2459f94a7a17-tuning-conf-dir\") pod \"multus-additional-cni-plugins-22clf\" (UID: \"cccb57a5-0894-4319-9eeb-2459f94a7a17\") " pod="openshift-multus/multus-additional-cni-plugins-22clf"
May 29 02:52:30.848917 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848156    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d9a08de9-bea6-4561-8eae-62421b7584a9-ovnkube-script-lib\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.849697 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848170    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cccb57a5-0894-4319-9eeb-2459f94a7a17-system-cni-dir\") pod \"multus-additional-cni-plugins-22clf\" (UID: \"cccb57a5-0894-4319-9eeb-2459f94a7a17\") " pod="openshift-multus/multus-additional-cni-plugins-22clf"
May 29 02:52:30.849697 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848209    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-22dg5\" (UniqueName: \"kubernetes.io/projected/d9a08de9-bea6-4561-8eae-62421b7584a9-kube-api-access-22dg5\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.849697 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848274    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k5nlf\" (UniqueName: \"kubernetes.io/projected/532da6c4-ac98-4477-88e3-021b3a046918-kube-api-access-k5nlf\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.849697 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848283    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cccb57a5-0894-4319-9eeb-2459f94a7a17-tuning-conf-dir\") pod \"multus-additional-cni-plugins-22clf\" (UID: \"cccb57a5-0894-4319-9eeb-2459f94a7a17\") " pod="openshift-multus/multus-additional-cni-plugins-22clf"
May 29 02:52:30.849697 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848316    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/deaa9f6f-56b4-4875-85fc-a2396ad84029-konnectivity-ca\") pod \"konnectivity-agent-h2vvt\" (UID: \"deaa9f6f-56b4-4875-85fc-a2396ad84029\") " pod="kube-system/konnectivity-agent-h2vvt"
May 29 02:52:30.849697 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848327    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j2vfm\" (UniqueName: \"kubernetes.io/projected/757d1101-4ef1-451f-b984-012545775d40-kube-api-access-j2vfm\") pod \"network-metrics-daemon-r4cd8\" (UID: \"757d1101-4ef1-451f-b984-012545775d40\") " pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:52:30.849697 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848607    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-run-openvswitch\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.849697 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848626    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-host-run-ovn-kubernetes\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.849697 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848641    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-system-cni-dir\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.849697 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848659    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-sys\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.849697 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848677    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0-device-dir\") pod \"aws-ebs-csi-driver-node-h7rh2\" (UID: \"adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2"
May 29 02:52:30.849697 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848665    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-run-openvswitch\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.849697 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848703    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-host-slash\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.849697 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848740    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-host-run-ovn-kubernetes\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.849697 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848750    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-system-cni-dir\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.849697 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848751    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-cnibin\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.849697 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848765    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-host-slash\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.850245 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848798    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d9a08de9-bea6-4561-8eae-62421b7584a9-ovnkube-script-lib\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.850245 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848806    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-cnibin\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.850245 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848823    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cccb57a5-0894-4319-9eeb-2459f94a7a17-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-22clf\" (UID: \"cccb57a5-0894-4319-9eeb-2459f94a7a17\") " pod="openshift-multus/multus-additional-cni-plugins-22clf"
May 29 02:52:30.850245 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848857    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqjfn\" (UniqueName: \"kubernetes.io/projected/32a9fc68-65c5-42d9-9121-6fa454c910f4-kube-api-access-sqjfn\") pod \"iptables-alerter-zb5lm\" (UID: \"32a9fc68-65c5-42d9-9121-6fa454c910f4\") " pod="openshift-network-operator/iptables-alerter-zb5lm"
May 29 02:52:30.850245 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848883    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d9a08de9-bea6-4561-8eae-62421b7584a9-env-overrides\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.850245 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848909    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-multus-socket-dir-parent\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.850245 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848935    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-host-var-lib-kubelet\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.850245 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.848960    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-etc-systemd\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.850245 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849009    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-lib-modules\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.850245 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849033    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0-socket-dir\") pod \"aws-ebs-csi-driver-node-h7rh2\" (UID: \"adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2"
May 29 02:52:30.850245 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849038    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-multus-socket-dir-parent\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.850245 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849058    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-host-var-lib-cni-multus\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.850245 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849094    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-hostroot\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.850245 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849146    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-hostroot\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.850245 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849144    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-host-var-lib-cni-multus\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.850245 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849173    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-host-var-lib-kubelet\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.850245 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849182    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cccb57a5-0894-4319-9eeb-2459f94a7a17-os-release\") pod \"multus-additional-cni-plugins-22clf\" (UID: \"cccb57a5-0894-4319-9eeb-2459f94a7a17\") " pod="openshift-multus/multus-additional-cni-plugins-22clf"
May 29 02:52:30.850801 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849224    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ghnr\" (UniqueName: \"kubernetes.io/projected/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-kube-api-access-7ghnr\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.850801 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849269    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cccb57a5-0894-4319-9eeb-2459f94a7a17-os-release\") pod \"multus-additional-cni-plugins-22clf\" (UID: \"cccb57a5-0894-4319-9eeb-2459f94a7a17\") " pod="openshift-multus/multus-additional-cni-plugins-22clf"
May 29 02:52:30.850801 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849304    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cccb57a5-0894-4319-9eeb-2459f94a7a17-cnibin\") pod \"multus-additional-cni-plugins-22clf\" (UID: \"cccb57a5-0894-4319-9eeb-2459f94a7a17\") " pod="openshift-multus/multus-additional-cni-plugins-22clf"
May 29 02:52:30.850801 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849329    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cccb57a5-0894-4319-9eeb-2459f94a7a17-cni-binary-copy\") pod \"multus-additional-cni-plugins-22clf\" (UID: \"cccb57a5-0894-4319-9eeb-2459f94a7a17\") " pod="openshift-multus/multus-additional-cni-plugins-22clf"
May 29 02:52:30.850801 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849347    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d9a08de9-bea6-4561-8eae-62421b7584a9-env-overrides\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.850801 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849376    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cccb57a5-0894-4319-9eeb-2459f94a7a17-cnibin\") pod \"multus-additional-cni-plugins-22clf\" (UID: \"cccb57a5-0894-4319-9eeb-2459f94a7a17\") " pod="openshift-multus/multus-additional-cni-plugins-22clf"
May 29 02:52:30.850801 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849366    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cccb57a5-0894-4319-9eeb-2459f94a7a17-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-22clf\" (UID: \"cccb57a5-0894-4319-9eeb-2459f94a7a17\") " pod="openshift-multus/multus-additional-cni-plugins-22clf"
May 29 02:52:30.850801 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849399    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wckc4\" (UniqueName: \"kubernetes.io/projected/cccb57a5-0894-4319-9eeb-2459f94a7a17-kube-api-access-wckc4\") pod \"multus-additional-cni-plugins-22clf\" (UID: \"cccb57a5-0894-4319-9eeb-2459f94a7a17\") " pod="openshift-multus/multus-additional-cni-plugins-22clf"
May 29 02:52:30.850801 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849429    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/235d8ad4-77e9-477a-b8d4-4edc46151e4c-serviceca\") pod \"node-ca-92nfp\" (UID: \"235d8ad4-77e9-477a-b8d4-4edc46151e4c\") " pod="openshift-image-registry/node-ca-92nfp"
May 29 02:52:30.850801 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849455    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/846260de-5ed8-4c80-ad40-8480a09eddd2-tmp-dir\") pod \"node-resolver-wz44t\" (UID: \"846260de-5ed8-4c80-ad40-8480a09eddd2\") " pod="openshift-dns/node-resolver-wz44t"
May 29 02:52:30.850801 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849490    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-host-run-netns\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.850801 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849519    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-etc-openvswitch\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.850801 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849546    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-multus-conf-dir\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.850801 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849571    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/532da6c4-ac98-4477-88e3-021b3a046918-multus-daemon-config\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.850801 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849588    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-host-run-netns\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.850801 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849600    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0-registration-dir\") pod \"aws-ebs-csi-driver-node-h7rh2\" (UID: \"adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2"
May 29 02:52:30.850801 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849671    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/532da6c4-ac98-4477-88e3-021b3a046918-multus-conf-dir\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.851315 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849706    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d9a08de9-bea6-4561-8eae-62421b7584a9-etc-openvswitch\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.851315 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.849730    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cccb57a5-0894-4319-9eeb-2459f94a7a17-cni-binary-copy\") pod \"multus-additional-cni-plugins-22clf\" (UID: \"cccb57a5-0894-4319-9eeb-2459f94a7a17\") " pod="openshift-multus/multus-additional-cni-plugins-22clf"
May 29 02:52:30.851315 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.850080    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/532da6c4-ac98-4477-88e3-021b3a046918-multus-daemon-config\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.851315 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.850428    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d9a08de9-bea6-4561-8eae-62421b7584a9-ovn-node-metrics-cert\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.851315 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.850514    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/deaa9f6f-56b4-4875-85fc-a2396ad84029-agent-certs\") pod \"konnectivity-agent-h2vvt\" (UID: \"deaa9f6f-56b4-4875-85fc-a2396ad84029\") " pod="kube-system/konnectivity-agent-h2vvt"
May 29 02:52:30.853396 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:30.853370    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10985281f2f37314cf306f08a4b9cf5b.slice/crio-2e038412c1c764ad9105c08d0721cc54863bb9880b22c532dfe4affe1fcd599f WatchSource:0}: Error finding container 2e038412c1c764ad9105c08d0721cc54863bb9880b22c532dfe4affe1fcd599f: Status 404 returned error can't find the container with id 2e038412c1c764ad9105c08d0721cc54863bb9880b22c532dfe4affe1fcd599f
May 29 02:52:30.855111 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.855090    2559 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 02:52:30.856641 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.856622    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5nlf\" (UniqueName: \"kubernetes.io/projected/532da6c4-ac98-4477-88e3-021b3a046918-kube-api-access-k5nlf\") pod \"multus-7fhjp\" (UID: \"532da6c4-ac98-4477-88e3-021b3a046918\") " pod="openshift-multus/multus-7fhjp"
May 29 02:52:30.856641 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.856639    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 29 02:52:30.857006 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.856990    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wckc4\" (UniqueName: \"kubernetes.io/projected/cccb57a5-0894-4319-9eeb-2459f94a7a17-kube-api-access-wckc4\") pod \"multus-additional-cni-plugins-22clf\" (UID: \"cccb57a5-0894-4319-9eeb-2459f94a7a17\") " pod="openshift-multus/multus-additional-cni-plugins-22clf"
May 29 02:52:30.857141 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.857120    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2vfm\" (UniqueName: \"kubernetes.io/projected/757d1101-4ef1-451f-b984-012545775d40-kube-api-access-j2vfm\") pod \"network-metrics-daemon-r4cd8\" (UID: \"757d1101-4ef1-451f-b984-012545775d40\") " pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:52:30.857524 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.857480    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22dg5\" (UniqueName: \"kubernetes.io/projected/d9a08de9-bea6-4561-8eae-62421b7584a9-kube-api-access-22dg5\") pod \"ovnkube-node-psj9n\" (UID: \"d9a08de9-bea6-4561-8eae-62421b7584a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:30.922782 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:30.922756    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcf3ae000999ff406cf9a785430af920.slice/crio-bdd55f1ecd259990965eb9246801894572b2653328ec3a1566c620c2996f2be8 WatchSource:0}: Error finding container bdd55f1ecd259990965eb9246801894572b2653328ec3a1566c620c2996f2be8: Status 404 returned error can't find the container with id bdd55f1ecd259990965eb9246801894572b2653328ec3a1566c620c2996f2be8
May 29 02:52:30.950840 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.950812    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sqjfn\" (UniqueName: \"kubernetes.io/projected/32a9fc68-65c5-42d9-9121-6fa454c910f4-kube-api-access-sqjfn\") pod \"iptables-alerter-zb5lm\" (UID: \"32a9fc68-65c5-42d9-9121-6fa454c910f4\") " pod="openshift-network-operator/iptables-alerter-zb5lm"
May 29 02:52:30.950840 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.950843    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-etc-systemd\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.951001 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.950860    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-lib-modules\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.951001 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.950928    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-etc-systemd\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.951076 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951004    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-lib-modules\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.951076 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951016    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0-socket-dir\") pod \"aws-ebs-csi-driver-node-h7rh2\" (UID: \"adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2"
May 29 02:52:30.951076 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951065    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7ghnr\" (UniqueName: \"kubernetes.io/projected/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-kube-api-access-7ghnr\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.951178 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951096    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/235d8ad4-77e9-477a-b8d4-4edc46151e4c-serviceca\") pod \"node-ca-92nfp\" (UID: \"235d8ad4-77e9-477a-b8d4-4edc46151e4c\") " pod="openshift-image-registry/node-ca-92nfp"
May 29 02:52:30.951178 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951130    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/846260de-5ed8-4c80-ad40-8480a09eddd2-tmp-dir\") pod \"node-resolver-wz44t\" (UID: \"846260de-5ed8-4c80-ad40-8480a09eddd2\") " pod="openshift-dns/node-resolver-wz44t"
May 29 02:52:30.951178 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951162    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0-registration-dir\") pod \"aws-ebs-csi-driver-node-h7rh2\" (UID: \"adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2"
May 29 02:52:30.951308 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951178    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0-socket-dir\") pod \"aws-ebs-csi-driver-node-h7rh2\" (UID: \"adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2"
May 29 02:52:30.951308 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951191    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-etc-sysconfig\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.951308 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951216    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/32a9fc68-65c5-42d9-9121-6fa454c910f4-iptables-alerter-script\") pod \"iptables-alerter-zb5lm\" (UID: \"32a9fc68-65c5-42d9-9121-6fa454c910f4\") " pod="openshift-network-operator/iptables-alerter-zb5lm"
May 29 02:52:30.951308 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951230    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0-registration-dir\") pod \"aws-ebs-csi-driver-node-h7rh2\" (UID: \"adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2"
May 29 02:52:30.951308 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951241    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/32a9fc68-65c5-42d9-9121-6fa454c910f4-host-slash\") pod \"iptables-alerter-zb5lm\" (UID: \"32a9fc68-65c5-42d9-9121-6fa454c910f4\") " pod="openshift-network-operator/iptables-alerter-zb5lm"
May 29 02:52:30.951308 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951262    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-etc-sysconfig\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.951308 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951269    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h7rh2\" (UID: \"adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2"
May 29 02:52:30.951308 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951298    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0-etc-selinux\") pod \"aws-ebs-csi-driver-node-h7rh2\" (UID: \"adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2"
May 29 02:52:30.951664 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951328    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-var-lib-kubelet\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.951664 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951337    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/32a9fc68-65c5-42d9-9121-6fa454c910f4-host-slash\") pod \"iptables-alerter-zb5lm\" (UID: \"32a9fc68-65c5-42d9-9121-6fa454c910f4\") " pod="openshift-network-operator/iptables-alerter-zb5lm"
May 29 02:52:30.951664 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951355    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-tmp\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.951664 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951397    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-etc-sysctl-conf\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.951664 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951423    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vw95k\" (UniqueName: \"kubernetes.io/projected/235d8ad4-77e9-477a-b8d4-4edc46151e4c-kube-api-access-vw95k\") pod \"node-ca-92nfp\" (UID: \"235d8ad4-77e9-477a-b8d4-4edc46151e4c\") " pod="openshift-image-registry/node-ca-92nfp"
May 29 02:52:30.951664 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951436    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0-etc-selinux\") pod \"aws-ebs-csi-driver-node-h7rh2\" (UID: \"adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2"
May 29 02:52:30.951664 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951450    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-etc-kubernetes\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.951664 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951475    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-etc-tuned\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.951664 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951482    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h7rh2\" (UID: \"adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2"
May 29 02:52:30.951664 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951487    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/846260de-5ed8-4c80-ad40-8480a09eddd2-tmp-dir\") pod \"node-resolver-wz44t\" (UID: \"846260de-5ed8-4c80-ad40-8480a09eddd2\") " pod="openshift-dns/node-resolver-wz44t"
May 29 02:52:30.951664 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951502    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0-sys-fs\") pod \"aws-ebs-csi-driver-node-h7rh2\" (UID: \"adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2"
May 29 02:52:30.951664 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951515    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/235d8ad4-77e9-477a-b8d4-4edc46151e4c-serviceca\") pod \"node-ca-92nfp\" (UID: \"235d8ad4-77e9-477a-b8d4-4edc46151e4c\") " pod="openshift-image-registry/node-ca-92nfp"
May 29 02:52:30.951664 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951534    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-host\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.951664 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951537    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-var-lib-kubelet\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.951664 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951563    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hrsj8\" (UniqueName: \"kubernetes.io/projected/adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0-kube-api-access-hrsj8\") pod \"aws-ebs-csi-driver-node-h7rh2\" (UID: \"adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2"
May 29 02:52:30.951664 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951585    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-etc-kubernetes\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.951664 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951590    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/846260de-5ed8-4c80-ad40-8480a09eddd2-hosts-file\") pod \"node-resolver-wz44t\" (UID: \"846260de-5ed8-4c80-ad40-8480a09eddd2\") " pod="openshift-dns/node-resolver-wz44t"
May 29 02:52:30.952343 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951622    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zw6qf\" (UniqueName: \"kubernetes.io/projected/902df08f-86be-4b22-8929-a7087c7b2c9a-kube-api-access-zw6qf\") pod \"network-check-target-fnzpl\" (UID: \"902df08f-86be-4b22-8929-a7087c7b2c9a\") " pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:52:30.952343 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951647    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-run\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.952343 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951660    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/32a9fc68-65c5-42d9-9121-6fa454c910f4-iptables-alerter-script\") pod \"iptables-alerter-zb5lm\" (UID: \"32a9fc68-65c5-42d9-9121-6fa454c910f4\") " pod="openshift-network-operator/iptables-alerter-zb5lm"
May 29 02:52:30.952343 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951646    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-etc-sysctl-conf\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.952343 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951692    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-host\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.952343 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951726    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/846260de-5ed8-4c80-ad40-8480a09eddd2-hosts-file\") pod \"node-resolver-wz44t\" (UID: \"846260de-5ed8-4c80-ad40-8480a09eddd2\") " pod="openshift-dns/node-resolver-wz44t"
May 29 02:52:30.952343 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951726    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xdx6c\" (UniqueName: \"kubernetes.io/projected/846260de-5ed8-4c80-ad40-8480a09eddd2-kube-api-access-xdx6c\") pod \"node-resolver-wz44t\" (UID: \"846260de-5ed8-4c80-ad40-8480a09eddd2\") " pod="openshift-dns/node-resolver-wz44t"
May 29 02:52:30.952343 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951757    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-etc-modprobe-d\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.952343 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951816    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0-sys-fs\") pod \"aws-ebs-csi-driver-node-h7rh2\" (UID: \"adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2"
May 29 02:52:30.952343 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951836    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-run\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.952343 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951874    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/235d8ad4-77e9-477a-b8d4-4edc46151e4c-host\") pod \"node-ca-92nfp\" (UID: \"235d8ad4-77e9-477a-b8d4-4edc46151e4c\") " pod="openshift-image-registry/node-ca-92nfp"
May 29 02:52:30.952343 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951900    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-etc-sysctl-d\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.952343 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951933    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-sys\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.952343 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951950    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-etc-modprobe-d\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.952343 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951959    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0-device-dir\") pod \"aws-ebs-csi-driver-node-h7rh2\" (UID: \"adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2"
May 29 02:52:30.952343 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.951982    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/235d8ad4-77e9-477a-b8d4-4edc46151e4c-host\") pod \"node-ca-92nfp\" (UID: \"235d8ad4-77e9-477a-b8d4-4edc46151e4c\") " pod="openshift-image-registry/node-ca-92nfp"
May 29 02:52:30.952343 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.952047    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0-device-dir\") pod \"aws-ebs-csi-driver-node-h7rh2\" (UID: \"adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2"
May 29 02:52:30.952343 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.952058    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-etc-sysctl-d\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.952843 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.952085    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-sys\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.953553 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.953535    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-tmp\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.953710 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.953693    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-etc-tuned\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.957566 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:30.957550    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 02:52:30.957675 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:30.957572    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 02:52:30.957675 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:30.957585    2559 projected.go:194] Error preparing data for projected volume kube-api-access-zw6qf for pod openshift-network-diagnostics/network-check-target-fnzpl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:52:30.957675 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:30.957649    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/902df08f-86be-4b22-8929-a7087c7b2c9a-kube-api-access-zw6qf podName:902df08f-86be-4b22-8929-a7087c7b2c9a nodeName:}" failed. No retries permitted until 2026-05-29 02:52:31.457631582 +0000 UTC m=+2.212946640 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zw6qf" (UniqueName: "kubernetes.io/projected/902df08f-86be-4b22-8929-a7087c7b2c9a-kube-api-access-zw6qf") pod "network-check-target-fnzpl" (UID: "902df08f-86be-4b22-8929-a7087c7b2c9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:52:30.959448 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.959419    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrsj8\" (UniqueName: \"kubernetes.io/projected/adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0-kube-api-access-hrsj8\") pod \"aws-ebs-csi-driver-node-h7rh2\" (UID: \"adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2"
May 29 02:52:30.959872 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.959854    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw95k\" (UniqueName: \"kubernetes.io/projected/235d8ad4-77e9-477a-b8d4-4edc46151e4c-kube-api-access-vw95k\") pod \"node-ca-92nfp\" (UID: \"235d8ad4-77e9-477a-b8d4-4edc46151e4c\") " pod="openshift-image-registry/node-ca-92nfp"
May 29 02:52:30.960169 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.960153    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ghnr\" (UniqueName: \"kubernetes.io/projected/aa6f0c45-98c6-473a-af3b-51b04f61b1c9-kube-api-access-7ghnr\") pod \"tuned-c8wpt\" (UID: \"aa6f0c45-98c6-473a-af3b-51b04f61b1c9\") " pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:30.960393 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.960379    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqjfn\" (UniqueName: \"kubernetes.io/projected/32a9fc68-65c5-42d9-9121-6fa454c910f4-kube-api-access-sqjfn\") pod \"iptables-alerter-zb5lm\" (UID: \"32a9fc68-65c5-42d9-9121-6fa454c910f4\") " pod="openshift-network-operator/iptables-alerter-zb5lm"
May 29 02:52:30.960488 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:30.960472    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdx6c\" (UniqueName: \"kubernetes.io/projected/846260de-5ed8-4c80-ad40-8480a09eddd2-kube-api-access-xdx6c\") pod \"node-resolver-wz44t\" (UID: \"846260de-5ed8-4c80-ad40-8480a09eddd2\") " pod="openshift-dns/node-resolver-wz44t"
May 29 02:52:31.067443 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:31.067372    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h2vvt"
May 29 02:52:31.074040 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:31.074018    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeaa9f6f_56b4_4875_85fc_a2396ad84029.slice/crio-09e3313f0e21a5555f963e6b47975e1dd95a686fba639371ce1df2394be3d97e WatchSource:0}: Error finding container 09e3313f0e21a5555f963e6b47975e1dd95a686fba639371ce1df2394be3d97e: Status 404 returned error can't find the container with id 09e3313f0e21a5555f963e6b47975e1dd95a686fba639371ce1df2394be3d97e
May 29 02:52:31.079872 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:31.079854    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:31.085727 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:31.085705    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9a08de9_bea6_4561_8eae_62421b7584a9.slice/crio-490def2156888ad387163a9ceb4382274525dd22cd63582da92294a12ae6d272 WatchSource:0}: Error finding container 490def2156888ad387163a9ceb4382274525dd22cd63582da92294a12ae6d272: Status 404 returned error can't find the container with id 490def2156888ad387163a9ceb4382274525dd22cd63582da92294a12ae6d272
May 29 02:52:31.101472 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:31.101450    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7fhjp"
May 29 02:52:31.106428 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:31.106408    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod532da6c4_ac98_4477_88e3_021b3a046918.slice/crio-d68a4a44c764753dd355260d3d3b21cb6ba19372ffa9ad3dcd3337702d995192 WatchSource:0}: Error finding container d68a4a44c764753dd355260d3d3b21cb6ba19372ffa9ad3dcd3337702d995192: Status 404 returned error can't find the container with id d68a4a44c764753dd355260d3d3b21cb6ba19372ffa9ad3dcd3337702d995192
May 29 02:52:31.114557 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:31.114540    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-22clf"
May 29 02:52:31.119554 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:31.119534    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcccb57a5_0894_4319_9eeb_2459f94a7a17.slice/crio-47de61a724541582cef83c057ad90e28058dd12f55ae98c066f9130f32353dac WatchSource:0}: Error finding container 47de61a724541582cef83c057ad90e28058dd12f55ae98c066f9130f32353dac: Status 404 returned error can't find the container with id 47de61a724541582cef83c057ad90e28058dd12f55ae98c066f9130f32353dac
May 29 02:52:31.120840 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:31.120823    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zb5lm"
May 29 02:52:31.125992 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:31.125956    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32a9fc68_65c5_42d9_9121_6fa454c910f4.slice/crio-fc27c30354a59ff42c9ca78cfb1288fe5485a202df35a4ac975b89ca5e873eeb WatchSource:0}: Error finding container fc27c30354a59ff42c9ca78cfb1288fe5485a202df35a4ac975b89ca5e873eeb: Status 404 returned error can't find the container with id fc27c30354a59ff42c9ca78cfb1288fe5485a202df35a4ac975b89ca5e873eeb
May 29 02:52:31.135716 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:31.135699    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2"
May 29 02:52:31.140672 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:31.140654    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadeaccdd_0cad_4d76_bf8b_9194c0bbbfa0.slice/crio-3021b4b7db237dfd2f078b8f761ede3076f7a7d4537255530a157d5e63eba69e WatchSource:0}: Error finding container 3021b4b7db237dfd2f078b8f761ede3076f7a7d4537255530a157d5e63eba69e: Status 404 returned error can't find the container with id 3021b4b7db237dfd2f078b8f761ede3076f7a7d4537255530a157d5e63eba69e
May 29 02:52:31.144308 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:31.144292    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c8wpt"
May 29 02:52:31.149062 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:31.149043    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa6f0c45_98c6_473a_af3b_51b04f61b1c9.slice/crio-cfedb0ca649b8824611232b44851a3a254d88012f04e9062d0151d0ac4461b26 WatchSource:0}: Error finding container cfedb0ca649b8824611232b44851a3a254d88012f04e9062d0151d0ac4461b26: Status 404 returned error can't find the container with id cfedb0ca649b8824611232b44851a3a254d88012f04e9062d0151d0ac4461b26
May 29 02:52:31.160996 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:31.160980    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-92nfp"
May 29 02:52:31.164457 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:31.164438    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wz44t"
May 29 02:52:31.166866 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:31.166826    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod235d8ad4_77e9_477a_b8d4_4edc46151e4c.slice/crio-b98ca24aa6ebf2f0f4dfdc4c6dc93b8ea07b52dfb0e2e21e9afce26801b2c34b WatchSource:0}: Error finding container b98ca24aa6ebf2f0f4dfdc4c6dc93b8ea07b52dfb0e2e21e9afce26801b2c34b: Status 404 returned error can't find the container with id b98ca24aa6ebf2f0f4dfdc4c6dc93b8ea07b52dfb0e2e21e9afce26801b2c34b
May 29 02:52:31.170511 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:52:31.170494    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod846260de_5ed8_4c80_ad40_8480a09eddd2.slice/crio-5ee0a88713acf0099e0d3e45da52b2cf16eb535c2d0335ef32cb0654cd2a7d5f WatchSource:0}: Error finding container 5ee0a88713acf0099e0d3e45da52b2cf16eb535c2d0335ef32cb0654cd2a7d5f: Status 404 returned error can't find the container with id 5ee0a88713acf0099e0d3e45da52b2cf16eb535c2d0335ef32cb0654cd2a7d5f
May 29 02:52:31.354466 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:31.354435    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs\") pod \"network-metrics-daemon-r4cd8\" (UID: \"757d1101-4ef1-451f-b984-012545775d40\") " pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:52:31.354623 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:31.354608    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:52:31.354686 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:31.354680    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs podName:757d1101-4ef1-451f-b984-012545775d40 nodeName:}" failed. No retries permitted until 2026-05-29 02:52:32.35466128 +0000 UTC m=+3.109976334 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs") pod "network-metrics-daemon-r4cd8" (UID: "757d1101-4ef1-451f-b984-012545775d40") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:52:31.540140 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:31.540109    2559 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 02:52:31.556197 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:31.556168    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zw6qf\" (UniqueName: \"kubernetes.io/projected/902df08f-86be-4b22-8929-a7087c7b2c9a-kube-api-access-zw6qf\") pod \"network-check-target-fnzpl\" (UID: \"902df08f-86be-4b22-8929-a7087c7b2c9a\") " pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:52:31.556357 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:31.556313    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 02:52:31.556357 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:31.556331    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 02:52:31.556357 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:31.556344    2559 projected.go:194] Error preparing data for projected volume kube-api-access-zw6qf for pod openshift-network-diagnostics/network-check-target-fnzpl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:52:31.556511 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:31.556444    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/902df08f-86be-4b22-8929-a7087c7b2c9a-kube-api-access-zw6qf podName:902df08f-86be-4b22-8929-a7087c7b2c9a nodeName:}" failed. No retries permitted until 2026-05-29 02:52:32.556380823 +0000 UTC m=+3.311695868 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zw6qf" (UniqueName: "kubernetes.io/projected/902df08f-86be-4b22-8929-a7087c7b2c9a-kube-api-access-zw6qf") pod "network-check-target-fnzpl" (UID: "902df08f-86be-4b22-8929-a7087c7b2c9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:52:31.793012 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:31.792910    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-28 02:47:30 +0000 UTC" deadline="2027-12-26 19:30:33.149258489 +0000 UTC"
May 29 02:52:31.793012 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:31.792946    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13840h38m1.356316665s"
May 29 02:52:31.869729 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:31.869669    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psj9n" event={"ID":"d9a08de9-bea6-4561-8eae-62421b7584a9","Type":"ContainerStarted","Data":"490def2156888ad387163a9ceb4382274525dd22cd63582da92294a12ae6d272"}
May 29 02:52:31.886707 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:31.886570    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h2vvt" event={"ID":"deaa9f6f-56b4-4875-85fc-a2396ad84029","Type":"ContainerStarted","Data":"09e3313f0e21a5555f963e6b47975e1dd95a686fba639371ce1df2394be3d97e"}
May 29 02:52:31.890926 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:31.890814    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-122.ec2.internal" event={"ID":"fcf3ae000999ff406cf9a785430af920","Type":"ContainerStarted","Data":"bdd55f1ecd259990965eb9246801894572b2653328ec3a1566c620c2996f2be8"}
May 29 02:52:31.891840 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:31.891817    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-122.ec2.internal" event={"ID":"10985281f2f37314cf306f08a4b9cf5b","Type":"ContainerStarted","Data":"2e038412c1c764ad9105c08d0721cc54863bb9880b22c532dfe4affe1fcd599f"}
May 29 02:52:31.892959 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:31.892820    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wz44t" event={"ID":"846260de-5ed8-4c80-ad40-8480a09eddd2","Type":"ContainerStarted","Data":"5ee0a88713acf0099e0d3e45da52b2cf16eb535c2d0335ef32cb0654cd2a7d5f"}
May 29 02:52:31.904586 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:31.904554    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c8wpt" event={"ID":"aa6f0c45-98c6-473a-af3b-51b04f61b1c9","Type":"ContainerStarted","Data":"cfedb0ca649b8824611232b44851a3a254d88012f04e9062d0151d0ac4461b26"}
May 29 02:52:31.922490 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:31.922460    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2" event={"ID":"adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0","Type":"ContainerStarted","Data":"3021b4b7db237dfd2f078b8f761ede3076f7a7d4537255530a157d5e63eba69e"}
May 29 02:52:31.936410 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:31.936349    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zb5lm" event={"ID":"32a9fc68-65c5-42d9-9121-6fa454c910f4","Type":"ContainerStarted","Data":"fc27c30354a59ff42c9ca78cfb1288fe5485a202df35a4ac975b89ca5e873eeb"}
May 29 02:52:31.942475 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:31.942421    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-22clf" event={"ID":"cccb57a5-0894-4319-9eeb-2459f94a7a17","Type":"ContainerStarted","Data":"47de61a724541582cef83c057ad90e28058dd12f55ae98c066f9130f32353dac"}
May 29 02:52:31.943832 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:31.943810    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7fhjp" event={"ID":"532da6c4-ac98-4477-88e3-021b3a046918","Type":"ContainerStarted","Data":"d68a4a44c764753dd355260d3d3b21cb6ba19372ffa9ad3dcd3337702d995192"}
May 29 02:52:31.969464 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:31.969438    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-92nfp" event={"ID":"235d8ad4-77e9-477a-b8d4-4edc46151e4c","Type":"ContainerStarted","Data":"b98ca24aa6ebf2f0f4dfdc4c6dc93b8ea07b52dfb0e2e21e9afce26801b2c34b"}
May 29 02:52:32.226764 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:32.226731    2559 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 02:52:32.362049 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:32.362013    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs\") pod \"network-metrics-daemon-r4cd8\" (UID: \"757d1101-4ef1-451f-b984-012545775d40\") " pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:52:32.362244 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:32.362184    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:52:32.362303 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:32.362264    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs podName:757d1101-4ef1-451f-b984-012545775d40 nodeName:}" failed. No retries permitted until 2026-05-29 02:52:34.362242155 +0000 UTC m=+5.117557199 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs") pod "network-metrics-daemon-r4cd8" (UID: "757d1101-4ef1-451f-b984-012545775d40") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:52:32.563208 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:32.563124    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zw6qf\" (UniqueName: \"kubernetes.io/projected/902df08f-86be-4b22-8929-a7087c7b2c9a-kube-api-access-zw6qf\") pod \"network-check-target-fnzpl\" (UID: \"902df08f-86be-4b22-8929-a7087c7b2c9a\") " pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:52:32.563372 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:32.563326    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 02:52:32.563372 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:32.563344    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 02:52:32.563372 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:32.563357    2559 projected.go:194] Error preparing data for projected volume kube-api-access-zw6qf for pod openshift-network-diagnostics/network-check-target-fnzpl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:52:32.563528 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:32.563414    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/902df08f-86be-4b22-8929-a7087c7b2c9a-kube-api-access-zw6qf podName:902df08f-86be-4b22-8929-a7087c7b2c9a nodeName:}" failed. No retries permitted until 2026-05-29 02:52:34.563394943 +0000 UTC m=+5.318710010 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zw6qf" (UniqueName: "kubernetes.io/projected/902df08f-86be-4b22-8929-a7087c7b2c9a-kube-api-access-zw6qf") pod "network-check-target-fnzpl" (UID: "902df08f-86be-4b22-8929-a7087c7b2c9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:52:32.671570 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:32.671405    2559 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 02:52:32.793675 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:32.793636    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-28 02:47:30 +0000 UTC" deadline="2028-02-15 08:29:07.369541498 +0000 UTC"
May 29 02:52:32.793675 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:32.793673    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15053h36m34.5758726s"
May 29 02:52:32.844718 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:32.844637    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:52:32.844860 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:32.844755    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnzpl" podUID="902df08f-86be-4b22-8929-a7087c7b2c9a"
May 29 02:52:32.845172 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:32.845151    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:52:32.845296 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:32.845274    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4cd8" podUID="757d1101-4ef1-451f-b984-012545775d40"
May 29 02:52:34.378315 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:34.378277    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs\") pod \"network-metrics-daemon-r4cd8\" (UID: \"757d1101-4ef1-451f-b984-012545775d40\") " pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:52:34.378751 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:34.378441    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:52:34.378751 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:34.378503    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs podName:757d1101-4ef1-451f-b984-012545775d40 nodeName:}" failed. No retries permitted until 2026-05-29 02:52:38.378483418 +0000 UTC m=+9.133798462 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs") pod "network-metrics-daemon-r4cd8" (UID: "757d1101-4ef1-451f-b984-012545775d40") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:52:34.580248 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:34.579609    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zw6qf\" (UniqueName: \"kubernetes.io/projected/902df08f-86be-4b22-8929-a7087c7b2c9a-kube-api-access-zw6qf\") pod \"network-check-target-fnzpl\" (UID: \"902df08f-86be-4b22-8929-a7087c7b2c9a\") " pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:52:34.580248 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:34.579775    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 02:52:34.580248 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:34.579793    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 02:52:34.580248 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:34.579805    2559 projected.go:194] Error preparing data for projected volume kube-api-access-zw6qf for pod openshift-network-diagnostics/network-check-target-fnzpl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:52:34.580248 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:34.579865    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/902df08f-86be-4b22-8929-a7087c7b2c9a-kube-api-access-zw6qf podName:902df08f-86be-4b22-8929-a7087c7b2c9a nodeName:}" failed. No retries permitted until 2026-05-29 02:52:38.579847394 +0000 UTC m=+9.335162448 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zw6qf" (UniqueName: "kubernetes.io/projected/902df08f-86be-4b22-8929-a7087c7b2c9a-kube-api-access-zw6qf") pod "network-check-target-fnzpl" (UID: "902df08f-86be-4b22-8929-a7087c7b2c9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:52:34.844834 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:34.844580    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:52:34.844988 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:34.844910    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnzpl" podUID="902df08f-86be-4b22-8929-a7087c7b2c9a"
May 29 02:52:34.845263 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:34.844726    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:52:34.845404 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:34.845363    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4cd8" podUID="757d1101-4ef1-451f-b984-012545775d40"
May 29 02:52:36.845027 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:36.844996    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:52:36.845426 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:36.845006    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:52:36.845426 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:36.845124    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnzpl" podUID="902df08f-86be-4b22-8929-a7087c7b2c9a"
May 29 02:52:36.845426 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:36.845169    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4cd8" podUID="757d1101-4ef1-451f-b984-012545775d40"
May 29 02:52:38.412764 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:38.412725    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs\") pod \"network-metrics-daemon-r4cd8\" (UID: \"757d1101-4ef1-451f-b984-012545775d40\") " pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:52:38.413249 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:38.412878    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:52:38.413249 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:38.412956    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs podName:757d1101-4ef1-451f-b984-012545775d40 nodeName:}" failed. No retries permitted until 2026-05-29 02:52:46.412934995 +0000 UTC m=+17.168250055 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs") pod "network-metrics-daemon-r4cd8" (UID: "757d1101-4ef1-451f-b984-012545775d40") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:52:38.614331 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:38.614279    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zw6qf\" (UniqueName: \"kubernetes.io/projected/902df08f-86be-4b22-8929-a7087c7b2c9a-kube-api-access-zw6qf\") pod \"network-check-target-fnzpl\" (UID: \"902df08f-86be-4b22-8929-a7087c7b2c9a\") " pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:52:38.614498 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:38.614461    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 02:52:38.614498 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:38.614483    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 02:52:38.614498 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:38.614496    2559 projected.go:194] Error preparing data for projected volume kube-api-access-zw6qf for pod openshift-network-diagnostics/network-check-target-fnzpl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:52:38.614652 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:38.614561    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/902df08f-86be-4b22-8929-a7087c7b2c9a-kube-api-access-zw6qf podName:902df08f-86be-4b22-8929-a7087c7b2c9a nodeName:}" failed. No retries permitted until 2026-05-29 02:52:46.614543737 +0000 UTC m=+17.369858782 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zw6qf" (UniqueName: "kubernetes.io/projected/902df08f-86be-4b22-8929-a7087c7b2c9a-kube-api-access-zw6qf") pod "network-check-target-fnzpl" (UID: "902df08f-86be-4b22-8929-a7087c7b2c9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:52:38.844459 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:38.844341    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:52:38.844623 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:38.844341    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:52:38.844623 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:38.844580    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4cd8" podUID="757d1101-4ef1-451f-b984-012545775d40"
May 29 02:52:38.844623 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:38.844452    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnzpl" podUID="902df08f-86be-4b22-8929-a7087c7b2c9a"
May 29 02:52:40.845070 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:40.845034    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:52:40.845501 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:40.845034    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:52:40.845501 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:40.845175    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4cd8" podUID="757d1101-4ef1-451f-b984-012545775d40"
May 29 02:52:40.845501 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:40.845251    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnzpl" podUID="902df08f-86be-4b22-8929-a7087c7b2c9a"
May 29 02:52:42.844465 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:42.844430    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:52:42.844897 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:42.844431    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:52:42.844897 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:42.844551    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4cd8" podUID="757d1101-4ef1-451f-b984-012545775d40"
May 29 02:52:42.844897 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:42.844636    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnzpl" podUID="902df08f-86be-4b22-8929-a7087c7b2c9a"
May 29 02:52:44.844571 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:44.844535    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:52:44.844571 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:44.844551    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:52:44.845037 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:44.844651    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnzpl" podUID="902df08f-86be-4b22-8929-a7087c7b2c9a"
May 29 02:52:44.845037 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:44.844797    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4cd8" podUID="757d1101-4ef1-451f-b984-012545775d40"
May 29 02:52:46.472077 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:46.472046    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs\") pod \"network-metrics-daemon-r4cd8\" (UID: \"757d1101-4ef1-451f-b984-012545775d40\") " pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:52:46.472496 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:46.472169    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:52:46.472496 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:46.472227    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs podName:757d1101-4ef1-451f-b984-012545775d40 nodeName:}" failed. No retries permitted until 2026-05-29 02:53:02.472213215 +0000 UTC m=+33.227528258 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs") pod "network-metrics-daemon-r4cd8" (UID: "757d1101-4ef1-451f-b984-012545775d40") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:52:46.673882 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:46.673848    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zw6qf\" (UniqueName: \"kubernetes.io/projected/902df08f-86be-4b22-8929-a7087c7b2c9a-kube-api-access-zw6qf\") pod \"network-check-target-fnzpl\" (UID: \"902df08f-86be-4b22-8929-a7087c7b2c9a\") " pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:52:46.674048 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:46.673994    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 02:52:46.674048 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:46.674008    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 02:52:46.674048 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:46.674017    2559 projected.go:194] Error preparing data for projected volume kube-api-access-zw6qf for pod openshift-network-diagnostics/network-check-target-fnzpl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:52:46.674190 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:46.674079    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/902df08f-86be-4b22-8929-a7087c7b2c9a-kube-api-access-zw6qf podName:902df08f-86be-4b22-8929-a7087c7b2c9a nodeName:}" failed. No retries permitted until 2026-05-29 02:53:02.674062737 +0000 UTC m=+33.429377782 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zw6qf" (UniqueName: "kubernetes.io/projected/902df08f-86be-4b22-8929-a7087c7b2c9a-kube-api-access-zw6qf") pod "network-check-target-fnzpl" (UID: "902df08f-86be-4b22-8929-a7087c7b2c9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:52:46.845278 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:46.845209    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:52:46.845419 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:46.845315    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnzpl" podUID="902df08f-86be-4b22-8929-a7087c7b2c9a"
May 29 02:52:46.845419 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:46.845214    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:52:46.845503 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:46.845481    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4cd8" podUID="757d1101-4ef1-451f-b984-012545775d40"
May 29 02:52:48.844740 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:48.844703    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:52:48.845188 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:48.844703    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:52:48.845188 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:48.844845    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4cd8" podUID="757d1101-4ef1-451f-b984-012545775d40"
May 29 02:52:48.845188 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:48.844923    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnzpl" podUID="902df08f-86be-4b22-8929-a7087c7b2c9a"
May 29 02:52:50.844739 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:50.844557    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:52:50.845051 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:50.844840    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnzpl" podUID="902df08f-86be-4b22-8929-a7087c7b2c9a"
May 29 02:52:50.845051 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:50.844567    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:52:50.845051 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:50.845021    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4cd8" podUID="757d1101-4ef1-451f-b984-012545775d40"
May 29 02:52:51.003913 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:51.003886    2559 generic.go:358] "Generic (PLEG): container finished" podID="cccb57a5-0894-4319-9eeb-2459f94a7a17" containerID="ec79fd3a2e488dd5eceb169049ad6f19b5edf45021719af24b73ef8a5c71fa63" exitCode=0
May 29 02:52:51.004060 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:51.003953    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-22clf" event={"ID":"cccb57a5-0894-4319-9eeb-2459f94a7a17","Type":"ContainerDied","Data":"ec79fd3a2e488dd5eceb169049ad6f19b5edf45021719af24b73ef8a5c71fa63"}
May 29 02:52:51.005287 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:51.005261    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7fhjp" event={"ID":"532da6c4-ac98-4477-88e3-021b3a046918","Type":"ContainerStarted","Data":"f87e148196858cd9a00adc6ca6e286d2c9c524f53cbc54450a21b0486579853a"}
May 29 02:52:51.006555 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:51.006473    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-92nfp" event={"ID":"235d8ad4-77e9-477a-b8d4-4edc46151e4c","Type":"ContainerStarted","Data":"30f1699965c0b829174e08fb49c6213f62f3a94dcbd39b87c92b7f2818c9ca61"}
May 29 02:52:51.007702 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:51.007682    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psj9n" event={"ID":"d9a08de9-bea6-4561-8eae-62421b7584a9","Type":"ContainerStarted","Data":"84beefb1cc766fd5d79f383a33246950e2ce8403027ed50e30e24af80b67f827"}
May 29 02:52:51.008886 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:51.008854    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h2vvt" event={"ID":"deaa9f6f-56b4-4875-85fc-a2396ad84029","Type":"ContainerStarted","Data":"edec7c285bcf5ca57892ae00285285917c853d9f58fd0c7afcae11b06b6f80ca"}
May 29 02:52:51.010039 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:51.010016    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-122.ec2.internal" event={"ID":"fcf3ae000999ff406cf9a785430af920","Type":"ContainerStarted","Data":"fdfbf1995193f833467732edf46c7f461ab064d2aa01d54ce43f25a4002e10e2"}
May 29 02:52:51.011133 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:51.011114    2559 generic.go:358] "Generic (PLEG): container finished" podID="10985281f2f37314cf306f08a4b9cf5b" containerID="42bb196676a3f8ebdfeadad3623ad42efa7280c041b23e54fecf3d197e3a9671" exitCode=0
May 29 02:52:51.011202 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:51.011167    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-122.ec2.internal" event={"ID":"10985281f2f37314cf306f08a4b9cf5b","Type":"ContainerDied","Data":"42bb196676a3f8ebdfeadad3623ad42efa7280c041b23e54fecf3d197e3a9671"}
May 29 02:52:51.012265 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:51.012246    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wz44t" event={"ID":"846260de-5ed8-4c80-ad40-8480a09eddd2","Type":"ContainerStarted","Data":"aeddd0a92b21009a2d7edf15dcffe7c13195ea2842a9edc35e4bcfdf27226f17"}
May 29 02:52:51.013396 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:51.013378    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c8wpt" event={"ID":"aa6f0c45-98c6-473a-af3b-51b04f61b1c9","Type":"ContainerStarted","Data":"880479c61f3bc7f4b746c67ca3938e3afec2279eb3368864d38800069c726461"}
May 29 02:52:51.014598 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:51.014565    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2" event={"ID":"adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0","Type":"ContainerStarted","Data":"e60647ef1bc8a2e6cfd489fbe08d1f90dc19fd1ca7884d289e61c5f32205848e"}
May 29 02:52:51.035455 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:51.035411    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-h2vvt" podStartSLOduration=3.174995678 podStartE2EDuration="22.035402182s" podCreationTimestamp="2026-05-29 02:52:29 +0000 UTC" firstStartedPulling="2026-05-29 02:52:31.075446054 +0000 UTC m=+1.830761096" lastFinishedPulling="2026-05-29 02:52:49.935852555 +0000 UTC m=+20.691167600" observedRunningTime="2026-05-29 02:52:51.035251247 +0000 UTC m=+21.790566312" watchObservedRunningTime="2026-05-29 02:52:51.035402182 +0000 UTC m=+21.790717246"
May 29 02:52:51.047314 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:51.047279    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-92nfp" podStartSLOduration=2.209179179 podStartE2EDuration="21.047270862s" podCreationTimestamp="2026-05-29 02:52:30 +0000 UTC" firstStartedPulling="2026-05-29 02:52:31.167963271 +0000 UTC m=+1.923278325" lastFinishedPulling="2026-05-29 02:52:50.00605496 +0000 UTC m=+20.761370008" observedRunningTime="2026-05-29 02:52:51.046838365 +0000 UTC m=+21.802153428" watchObservedRunningTime="2026-05-29 02:52:51.047270862 +0000 UTC m=+21.802585927"
May 29 02:52:51.081198 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:51.081158    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-7fhjp" podStartSLOduration=2.149563798 podStartE2EDuration="21.081145201s" podCreationTimestamp="2026-05-29 02:52:30 +0000 UTC" firstStartedPulling="2026-05-29 02:52:31.107679282 +0000 UTC m=+1.862994324" lastFinishedPulling="2026-05-29 02:52:50.039260682 +0000 UTC m=+20.794575727" observedRunningTime="2026-05-29 02:52:51.06190623 +0000 UTC m=+21.817221293" watchObservedRunningTime="2026-05-29 02:52:51.081145201 +0000 UTC m=+21.836460266"
May 29 02:52:51.081588 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:51.081554    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-c8wpt" podStartSLOduration=2.195888038 podStartE2EDuration="21.081547801s" podCreationTimestamp="2026-05-29 02:52:30 +0000 UTC" firstStartedPulling="2026-05-29 02:52:31.150360777 +0000 UTC m=+1.905675819" lastFinishedPulling="2026-05-29 02:52:50.036020531 +0000 UTC m=+20.791335582" observedRunningTime="2026-05-29 02:52:51.081271655 +0000 UTC m=+21.836586712" watchObservedRunningTime="2026-05-29 02:52:51.081547801 +0000 UTC m=+21.836862865"
May 29 02:52:51.105692 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:51.105659    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-wz44t" podStartSLOduration=2.237645299 podStartE2EDuration="21.105649132s" podCreationTimestamp="2026-05-29 02:52:30 +0000 UTC" firstStartedPulling="2026-05-29 02:52:31.171848008 +0000 UTC m=+1.927163050" lastFinishedPulling="2026-05-29 02:52:50.039851838 +0000 UTC m=+20.795166883" observedRunningTime="2026-05-29 02:52:51.093944752 +0000 UTC m=+21.849259834" watchObservedRunningTime="2026-05-29 02:52:51.105649132 +0000 UTC m=+21.860964195"
May 29 02:52:52.019276 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:52.019092    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psj9n_d9a08de9-bea6-4561-8eae-62421b7584a9/ovn-acl-logging/0.log"
May 29 02:52:52.020071 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:52.019580    2559 generic.go:358] "Generic (PLEG): container finished" podID="d9a08de9-bea6-4561-8eae-62421b7584a9" containerID="e22c67e50daa669fd469a5bcd0194e1abd69c6a66188929ae86f3d4b90e10761" exitCode=1
May 29 02:52:52.020071 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:52.019656    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psj9n" event={"ID":"d9a08de9-bea6-4561-8eae-62421b7584a9","Type":"ContainerStarted","Data":"e64a834586a30456152d3c46454caec2816967c555d71f87b589079ca22b1f6a"}
May 29 02:52:52.020071 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:52.019688    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psj9n" event={"ID":"d9a08de9-bea6-4561-8eae-62421b7584a9","Type":"ContainerStarted","Data":"f676809a44516d93f9918e195e1a6b9d4fa5876fec61ababb9979f6cb16a9d6b"}
May 29 02:52:52.020071 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:52.019697    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psj9n" event={"ID":"d9a08de9-bea6-4561-8eae-62421b7584a9","Type":"ContainerStarted","Data":"b6c3a0ce1a87f923508c0dbc193debaa4379df59f858949ef00a28d3199400ca"}
May 29 02:52:52.020071 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:52.019706    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psj9n" event={"ID":"d9a08de9-bea6-4561-8eae-62421b7584a9","Type":"ContainerStarted","Data":"781e0353844350a946997a321a7b0643d5102c03fa577fd59880ef8df81deb70"}
May 29 02:52:52.020071 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:52.019715    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psj9n" event={"ID":"d9a08de9-bea6-4561-8eae-62421b7584a9","Type":"ContainerDied","Data":"e22c67e50daa669fd469a5bcd0194e1abd69c6a66188929ae86f3d4b90e10761"}
May 29 02:52:52.020911 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:52.020888    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zb5lm" event={"ID":"32a9fc68-65c5-42d9-9121-6fa454c910f4","Type":"ContainerStarted","Data":"f42651a45b4142f8b31ea8c7bd3dbd2d4082377eda8dc4ab83573e374d4b89c6"}
May 29 02:52:52.034591 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:52.034547    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-122.ec2.internal" podStartSLOduration=22.034530255 podStartE2EDuration="22.034530255s" podCreationTimestamp="2026-05-29 02:52:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 02:52:51.105649198 +0000 UTC m=+21.860964263" watchObservedRunningTime="2026-05-29 02:52:52.034530255 +0000 UTC m=+22.789845320"
May 29 02:52:52.034954 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:52.034919    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zb5lm" podStartSLOduration=3.128258595 podStartE2EDuration="22.034910687s" podCreationTimestamp="2026-05-29 02:52:30 +0000 UTC" firstStartedPulling="2026-05-29 02:52:31.127377194 +0000 UTC m=+1.882692239" lastFinishedPulling="2026-05-29 02:52:50.034029282 +0000 UTC m=+20.789344331" observedRunningTime="2026-05-29 02:52:52.034608125 +0000 UTC m=+22.789923190" watchObservedRunningTime="2026-05-29 02:52:52.034910687 +0000 UTC m=+22.790225748"
May 29 02:52:52.122896 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:52.122868    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-h2vvt"
May 29 02:52:52.123466 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:52.123446    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-h2vvt"
May 29 02:52:52.327788 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:52.327759    2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 29 02:52:52.812588 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:52.812475    2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-29T02:52:52.327777042Z","UUID":"ca2e9aa9-796e-4352-9c2c-eec8c0c03ace","Handler":null,"Name":"","Endpoint":""}
May 29 02:52:52.814172 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:52.814154    2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 29 02:52:52.814172 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:52.814175    2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 29 02:52:52.845140 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:52.845113    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:52:52.845140 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:52.845125    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:52:52.845331 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:52.845220    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnzpl" podUID="902df08f-86be-4b22-8929-a7087c7b2c9a"
May 29 02:52:52.845331 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:52.845311    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4cd8" podUID="757d1101-4ef1-451f-b984-012545775d40"
May 29 02:52:53.024821 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:53.024787    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2" event={"ID":"adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0","Type":"ContainerStarted","Data":"a563e5266e4013a7e1054317eb6447aa77a96b678f355dfc716e5bf2f290cbe0"}
May 29 02:52:53.026726 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:53.026699    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-122.ec2.internal" event={"ID":"10985281f2f37314cf306f08a4b9cf5b","Type":"ContainerStarted","Data":"4ed1f8cf34d774835ef795566ae94070aa3230d3d9558291f503cae1fa683f4a"}
May 29 02:52:53.041079 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:53.041045    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-122.ec2.internal" podStartSLOduration=23.041033597 podStartE2EDuration="23.041033597s" podCreationTimestamp="2026-05-29 02:52:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 02:52:53.040807121 +0000 UTC m=+23.796122189" watchObservedRunningTime="2026-05-29 02:52:53.041033597 +0000 UTC m=+23.796348661"
May 29 02:52:54.029082 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:54.028426    2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
May 29 02:52:54.844845 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:54.844816    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:52:54.845021 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:54.844858    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:52:54.845021 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:54.844916    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnzpl" podUID="902df08f-86be-4b22-8929-a7087c7b2c9a"
May 29 02:52:54.845136 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:54.845055    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4cd8" podUID="757d1101-4ef1-451f-b984-012545775d40"
May 29 02:52:56.033569 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:56.033532    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2" event={"ID":"adeaccdd-0cad-4d76-bf8b-9194c0bbbfa0","Type":"ContainerStarted","Data":"e4a840d53a0997a976598133c859492b8c7137bef63294ebf36dff5a15b799cb"}
May 29 02:52:56.035073 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:56.035050    2559 generic.go:358] "Generic (PLEG): container finished" podID="cccb57a5-0894-4319-9eeb-2459f94a7a17" containerID="03a49c08b88b73646f274ca603f1484e31d76dabfc44e0a7b72612e4740d5974" exitCode=0
May 29 02:52:56.035159 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:56.035135    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-22clf" event={"ID":"cccb57a5-0894-4319-9eeb-2459f94a7a17","Type":"ContainerDied","Data":"03a49c08b88b73646f274ca603f1484e31d76dabfc44e0a7b72612e4740d5974"}
May 29 02:52:56.037547 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:56.037529    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psj9n_d9a08de9-bea6-4561-8eae-62421b7584a9/ovn-acl-logging/0.log"
May 29 02:52:56.037816 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:56.037799    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psj9n" event={"ID":"d9a08de9-bea6-4561-8eae-62421b7584a9","Type":"ContainerStarted","Data":"4231c8a9340d19553c6ec90ee0e8e52879703214044aa16eda8290719a2d1f0d"}
May 29 02:52:56.049116 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:56.049079    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h7rh2" podStartSLOduration=2.148475531 podStartE2EDuration="26.0490687s" podCreationTimestamp="2026-05-29 02:52:30 +0000 UTC" firstStartedPulling="2026-05-29 02:52:31.141887367 +0000 UTC m=+1.897202412" lastFinishedPulling="2026-05-29 02:52:55.04248054 +0000 UTC m=+25.797795581" observedRunningTime="2026-05-29 02:52:56.048608834 +0000 UTC m=+26.803923899" watchObservedRunningTime="2026-05-29 02:52:56.0490687 +0000 UTC m=+26.804383763"
May 29 02:52:56.844516 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:56.844484    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:52:56.844662 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:56.844488    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:52:56.844662 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:56.844584    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnzpl" podUID="902df08f-86be-4b22-8929-a7087c7b2c9a"
May 29 02:52:56.844746 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:56.844666    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4cd8" podUID="757d1101-4ef1-451f-b984-012545775d40"
May 29 02:52:57.040893 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:57.040815    2559 generic.go:358] "Generic (PLEG): container finished" podID="cccb57a5-0894-4319-9eeb-2459f94a7a17" containerID="56d91afaa1c6b403827b806a35610b081d74d97bc595828daee25e0a3baf88ba" exitCode=0
May 29 02:52:57.041367 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:57.040908    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-22clf" event={"ID":"cccb57a5-0894-4319-9eeb-2459f94a7a17","Type":"ContainerDied","Data":"56d91afaa1c6b403827b806a35610b081d74d97bc595828daee25e0a3baf88ba"}
May 29 02:52:57.701543 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:57.701317    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-h2vvt"
May 29 02:52:57.701673 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:57.701661    2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
May 29 02:52:57.701910 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:57.701890    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-h2vvt"
May 29 02:52:58.045256 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:58.045228    2559 generic.go:358] "Generic (PLEG): container finished" podID="cccb57a5-0894-4319-9eeb-2459f94a7a17" containerID="6517c415532cdef330796109a5bddf5670929af4113fa4b07c44d399eb1d72de" exitCode=0
May 29 02:52:58.045686 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:58.045308    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-22clf" event={"ID":"cccb57a5-0894-4319-9eeb-2459f94a7a17","Type":"ContainerDied","Data":"6517c415532cdef330796109a5bddf5670929af4113fa4b07c44d399eb1d72de"}
May 29 02:52:58.048280 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:58.048263    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psj9n_d9a08de9-bea6-4561-8eae-62421b7584a9/ovn-acl-logging/0.log"
May 29 02:52:58.048650 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:58.048624    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psj9n" event={"ID":"d9a08de9-bea6-4561-8eae-62421b7584a9","Type":"ContainerStarted","Data":"85af4e90c0d30e605ad79fe9fe4af8f12ad14278ae273598ea3e89c1974d7bcc"}
May 29 02:52:58.048926 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:58.048905    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:58.049058 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:58.048933    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:58.049058 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:58.048946    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:58.049058 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:58.049028    2559 scope.go:117] "RemoveContainer" containerID="e22c67e50daa669fd469a5bcd0194e1abd69c6a66188929ae86f3d4b90e10761"
May 29 02:52:58.065106 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:58.065086    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:58.065198 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:58.065169    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:52:58.844708 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:58.844679    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:52:58.844905 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:58.844814    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnzpl" podUID="902df08f-86be-4b22-8929-a7087c7b2c9a"
May 29 02:52:58.844984 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:58.844927    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:52:58.845071 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:58.845050    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4cd8" podUID="757d1101-4ef1-451f-b984-012545775d40"
May 29 02:52:59.055068 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:59.055041    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psj9n_d9a08de9-bea6-4561-8eae-62421b7584a9/ovn-acl-logging/0.log"
May 29 02:52:59.055481 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:59.055382    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psj9n" event={"ID":"d9a08de9-bea6-4561-8eae-62421b7584a9","Type":"ContainerStarted","Data":"280397027f346431c364fdf31c010d2bb0e5db2234acf5a5666f928a817172f4"}
May 29 02:52:59.082663 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:59.082604    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-psj9n" podStartSLOduration=9.539613837 podStartE2EDuration="29.08258798s" podCreationTimestamp="2026-05-29 02:52:30 +0000 UTC" firstStartedPulling="2026-05-29 02:52:31.087374413 +0000 UTC m=+1.842689468" lastFinishedPulling="2026-05-29 02:52:50.630348556 +0000 UTC m=+21.385663611" observedRunningTime="2026-05-29 02:52:59.081139174 +0000 UTC m=+29.836454240" watchObservedRunningTime="2026-05-29 02:52:59.08258798 +0000 UTC m=+29.837903045"
May 29 02:52:59.152104 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:59.152070    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r4cd8"]
May 29 02:52:59.152252 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:59.152215    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:52:59.152339 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:59.152316    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4cd8" podUID="757d1101-4ef1-451f-b984-012545775d40"
May 29 02:52:59.155008 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:59.154925    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fnzpl"]
May 29 02:52:59.155152 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:52:59.155061    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:52:59.155236 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:52:59.155153    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnzpl" podUID="902df08f-86be-4b22-8929-a7087c7b2c9a"
May 29 02:53:00.844936 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:00.844729    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:53:00.845340 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:00.844759    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:53:00.845340 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:00.845061    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4cd8" podUID="757d1101-4ef1-451f-b984-012545775d40"
May 29 02:53:00.845340 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:00.845167    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnzpl" podUID="902df08f-86be-4b22-8929-a7087c7b2c9a"
May 29 02:53:02.493578 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:02.493537    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs\") pod \"network-metrics-daemon-r4cd8\" (UID: \"757d1101-4ef1-451f-b984-012545775d40\") " pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:53:02.494088 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:02.493698    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:53:02.494088 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:02.493767    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs podName:757d1101-4ef1-451f-b984-012545775d40 nodeName:}" failed. No retries permitted until 2026-05-29 02:53:34.493745543 +0000 UTC m=+65.249060584 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs") pod "network-metrics-daemon-r4cd8" (UID: "757d1101-4ef1-451f-b984-012545775d40") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:53:02.694897 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:02.694851    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zw6qf\" (UniqueName: \"kubernetes.io/projected/902df08f-86be-4b22-8929-a7087c7b2c9a-kube-api-access-zw6qf\") pod \"network-check-target-fnzpl\" (UID: \"902df08f-86be-4b22-8929-a7087c7b2c9a\") " pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:53:02.695093 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:02.695056    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 02:53:02.695093 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:02.695083    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 02:53:02.695188 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:02.695097    2559 projected.go:194] Error preparing data for projected volume kube-api-access-zw6qf for pod openshift-network-diagnostics/network-check-target-fnzpl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:53:02.695188 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:02.695161    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/902df08f-86be-4b22-8929-a7087c7b2c9a-kube-api-access-zw6qf podName:902df08f-86be-4b22-8929-a7087c7b2c9a nodeName:}" failed. No retries permitted until 2026-05-29 02:53:34.695142168 +0000 UTC m=+65.450457214 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zw6qf" (UniqueName: "kubernetes.io/projected/902df08f-86be-4b22-8929-a7087c7b2c9a-kube-api-access-zw6qf") pod "network-check-target-fnzpl" (UID: "902df08f-86be-4b22-8929-a7087c7b2c9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:53:02.844812 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:02.844730    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:53:02.845040 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:02.844730    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:53:02.845040 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:02.844858    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnzpl" podUID="902df08f-86be-4b22-8929-a7087c7b2c9a"
May 29 02:53:02.845040 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:02.844949    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4cd8" podUID="757d1101-4ef1-451f-b984-012545775d40"
May 29 02:53:03.261581 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.261550    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-122.ec2.internal" event="NodeReady"
May 29 02:53:03.261754 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.261675    2559 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 29 02:53:03.306434 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.306404    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9c6sj"]
May 29 02:53:03.330109 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.330082    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bzt86"]
May 29 02:53:03.330273 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.330256    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9c6sj"
May 29 02:53:03.332044 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.331922    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 29 02:53:03.332044 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.332001    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 29 02:53:03.332377 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.332320    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4sfm2\""
May 29 02:53:03.346348 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.346329    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9c6sj"]
May 29 02:53:03.346456 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.346355    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bzt86"]
May 29 02:53:03.346456 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.346451    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bzt86"
May 29 02:53:03.348355 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.348337    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2t8gv\""
May 29 02:53:03.348453 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.348360    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 29 02:53:03.348453 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.348383    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 29 02:53:03.348453 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.348415    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 29 02:53:03.401124 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.401078    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wxqw\" (UniqueName: \"kubernetes.io/projected/e7cca70d-3eb6-4647-9508-894b981687eb-kube-api-access-9wxqw\") pod \"dns-default-9c6sj\" (UID: \"e7cca70d-3eb6-4647-9508-894b981687eb\") " pod="openshift-dns/dns-default-9c6sj"
May 29 02:53:03.401243 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.401159    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7cca70d-3eb6-4647-9508-894b981687eb-tmp-dir\") pod \"dns-default-9c6sj\" (UID: \"e7cca70d-3eb6-4647-9508-894b981687eb\") " pod="openshift-dns/dns-default-9c6sj"
May 29 02:53:03.401243 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.401185    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e7cca70d-3eb6-4647-9508-894b981687eb-config-volume\") pod \"dns-default-9c6sj\" (UID: \"e7cca70d-3eb6-4647-9508-894b981687eb\") " pod="openshift-dns/dns-default-9c6sj"
May 29 02:53:03.401243 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.401203    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7cca70d-3eb6-4647-9508-894b981687eb-metrics-tls\") pod \"dns-default-9c6sj\" (UID: \"e7cca70d-3eb6-4647-9508-894b981687eb\") " pod="openshift-dns/dns-default-9c6sj"
May 29 02:53:03.501786 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.501746    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9wxqw\" (UniqueName: \"kubernetes.io/projected/e7cca70d-3eb6-4647-9508-894b981687eb-kube-api-access-9wxqw\") pod \"dns-default-9c6sj\" (UID: \"e7cca70d-3eb6-4647-9508-894b981687eb\") " pod="openshift-dns/dns-default-9c6sj"
May 29 02:53:03.502402 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.501807    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brnj7\" (UniqueName: \"kubernetes.io/projected/e5ebef60-aa2e-4a66-ad59-307085eae08c-kube-api-access-brnj7\") pod \"ingress-canary-bzt86\" (UID: \"e5ebef60-aa2e-4a66-ad59-307085eae08c\") " pod="openshift-ingress-canary/ingress-canary-bzt86"
May 29 02:53:03.502402 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.501838    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5ebef60-aa2e-4a66-ad59-307085eae08c-cert\") pod \"ingress-canary-bzt86\" (UID: \"e5ebef60-aa2e-4a66-ad59-307085eae08c\") " pod="openshift-ingress-canary/ingress-canary-bzt86"
May 29 02:53:03.502402 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.501864    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7cca70d-3eb6-4647-9508-894b981687eb-tmp-dir\") pod \"dns-default-9c6sj\" (UID: \"e7cca70d-3eb6-4647-9508-894b981687eb\") " pod="openshift-dns/dns-default-9c6sj"
May 29 02:53:03.502402 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.501890    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e7cca70d-3eb6-4647-9508-894b981687eb-config-volume\") pod \"dns-default-9c6sj\" (UID: \"e7cca70d-3eb6-4647-9508-894b981687eb\") " pod="openshift-dns/dns-default-9c6sj"
May 29 02:53:03.502402 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.501913    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7cca70d-3eb6-4647-9508-894b981687eb-metrics-tls\") pod \"dns-default-9c6sj\" (UID: \"e7cca70d-3eb6-4647-9508-894b981687eb\") " pod="openshift-dns/dns-default-9c6sj"
May 29 02:53:03.502402 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:03.502068    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 02:53:03.502402 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:03.502132    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7cca70d-3eb6-4647-9508-894b981687eb-metrics-tls podName:e7cca70d-3eb6-4647-9508-894b981687eb nodeName:}" failed. No retries permitted until 2026-05-29 02:53:04.002113124 +0000 UTC m=+34.757428181 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7cca70d-3eb6-4647-9508-894b981687eb-metrics-tls") pod "dns-default-9c6sj" (UID: "e7cca70d-3eb6-4647-9508-894b981687eb") : secret "dns-default-metrics-tls" not found
May 29 02:53:03.502402 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.502258    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7cca70d-3eb6-4647-9508-894b981687eb-tmp-dir\") pod \"dns-default-9c6sj\" (UID: \"e7cca70d-3eb6-4647-9508-894b981687eb\") " pod="openshift-dns/dns-default-9c6sj"
May 29 02:53:03.502703 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.502473    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e7cca70d-3eb6-4647-9508-894b981687eb-config-volume\") pod \"dns-default-9c6sj\" (UID: \"e7cca70d-3eb6-4647-9508-894b981687eb\") " pod="openshift-dns/dns-default-9c6sj"
May 29 02:53:03.513785 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.513722    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wxqw\" (UniqueName: \"kubernetes.io/projected/e7cca70d-3eb6-4647-9508-894b981687eb-kube-api-access-9wxqw\") pod \"dns-default-9c6sj\" (UID: \"e7cca70d-3eb6-4647-9508-894b981687eb\") " pod="openshift-dns/dns-default-9c6sj"
May 29 02:53:03.602827 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.602789    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-brnj7\" (UniqueName: \"kubernetes.io/projected/e5ebef60-aa2e-4a66-ad59-307085eae08c-kube-api-access-brnj7\") pod \"ingress-canary-bzt86\" (UID: \"e5ebef60-aa2e-4a66-ad59-307085eae08c\") " pod="openshift-ingress-canary/ingress-canary-bzt86"
May 29 02:53:03.603013 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.602841    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5ebef60-aa2e-4a66-ad59-307085eae08c-cert\") pod \"ingress-canary-bzt86\" (UID: \"e5ebef60-aa2e-4a66-ad59-307085eae08c\") " pod="openshift-ingress-canary/ingress-canary-bzt86"
May 29 02:53:03.603013 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:03.603004    2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 02:53:03.603123 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:03.603061    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5ebef60-aa2e-4a66-ad59-307085eae08c-cert podName:e5ebef60-aa2e-4a66-ad59-307085eae08c nodeName:}" failed. No retries permitted until 2026-05-29 02:53:04.103042559 +0000 UTC m=+34.858357608 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e5ebef60-aa2e-4a66-ad59-307085eae08c-cert") pod "ingress-canary-bzt86" (UID: "e5ebef60-aa2e-4a66-ad59-307085eae08c") : secret "canary-serving-cert" not found
May 29 02:53:03.615650 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:03.615619    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brnj7\" (UniqueName: \"kubernetes.io/projected/e5ebef60-aa2e-4a66-ad59-307085eae08c-kube-api-access-brnj7\") pod \"ingress-canary-bzt86\" (UID: \"e5ebef60-aa2e-4a66-ad59-307085eae08c\") " pod="openshift-ingress-canary/ingress-canary-bzt86"
May 29 02:53:04.005260 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:04.005238    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7cca70d-3eb6-4647-9508-894b981687eb-metrics-tls\") pod \"dns-default-9c6sj\" (UID: \"e7cca70d-3eb6-4647-9508-894b981687eb\") " pod="openshift-dns/dns-default-9c6sj"
May 29 02:53:04.005382 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:04.005340    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 02:53:04.005424 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:04.005387    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7cca70d-3eb6-4647-9508-894b981687eb-metrics-tls podName:e7cca70d-3eb6-4647-9508-894b981687eb nodeName:}" failed. No retries permitted until 2026-05-29 02:53:05.005375025 +0000 UTC m=+35.760690068 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7cca70d-3eb6-4647-9508-894b981687eb-metrics-tls") pod "dns-default-9c6sj" (UID: "e7cca70d-3eb6-4647-9508-894b981687eb") : secret "dns-default-metrics-tls" not found
May 29 02:53:04.066799 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:04.066772    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-22clf" event={"ID":"cccb57a5-0894-4319-9eeb-2459f94a7a17","Type":"ContainerStarted","Data":"21e152b1f8f0cf92179c175e51e1dc4f2f8b296b8f8d658cb0d651ee99648d0b"}
May 29 02:53:04.105584 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:04.105548    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5ebef60-aa2e-4a66-ad59-307085eae08c-cert\") pod \"ingress-canary-bzt86\" (UID: \"e5ebef60-aa2e-4a66-ad59-307085eae08c\") " pod="openshift-ingress-canary/ingress-canary-bzt86"
May 29 02:53:04.105728 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:04.105667    2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 02:53:04.105728 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:04.105717    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5ebef60-aa2e-4a66-ad59-307085eae08c-cert podName:e5ebef60-aa2e-4a66-ad59-307085eae08c nodeName:}" failed. No retries permitted until 2026-05-29 02:53:05.105704019 +0000 UTC m=+35.861019061 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e5ebef60-aa2e-4a66-ad59-307085eae08c-cert") pod "ingress-canary-bzt86" (UID: "e5ebef60-aa2e-4a66-ad59-307085eae08c") : secret "canary-serving-cert" not found
May 29 02:53:04.844885 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:04.844860    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:53:04.845634 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:04.844864    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:53:04.848400 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:04.848377    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 29 02:53:04.848680 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:04.848661    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 29 02:53:04.848791 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:04.848687    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 29 02:53:04.848791 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:04.848712    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-r7pnj\""
May 29 02:53:04.848791 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:04.848718    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n4b9g\""
May 29 02:53:05.011638 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:05.011602    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7cca70d-3eb6-4647-9508-894b981687eb-metrics-tls\") pod \"dns-default-9c6sj\" (UID: \"e7cca70d-3eb6-4647-9508-894b981687eb\") " pod="openshift-dns/dns-default-9c6sj"
May 29 02:53:05.011773 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:05.011718    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 02:53:05.011773 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:05.011769    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7cca70d-3eb6-4647-9508-894b981687eb-metrics-tls podName:e7cca70d-3eb6-4647-9508-894b981687eb nodeName:}" failed. No retries permitted until 2026-05-29 02:53:07.011755593 +0000 UTC m=+37.767070635 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7cca70d-3eb6-4647-9508-894b981687eb-metrics-tls") pod "dns-default-9c6sj" (UID: "e7cca70d-3eb6-4647-9508-894b981687eb") : secret "dns-default-metrics-tls" not found
May 29 02:53:05.071308 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:05.071280    2559 generic.go:358] "Generic (PLEG): container finished" podID="cccb57a5-0894-4319-9eeb-2459f94a7a17" containerID="21e152b1f8f0cf92179c175e51e1dc4f2f8b296b8f8d658cb0d651ee99648d0b" exitCode=0
May 29 02:53:05.071413 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:05.071318    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-22clf" event={"ID":"cccb57a5-0894-4319-9eeb-2459f94a7a17","Type":"ContainerDied","Data":"21e152b1f8f0cf92179c175e51e1dc4f2f8b296b8f8d658cb0d651ee99648d0b"}
May 29 02:53:05.112766 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:05.112741    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5ebef60-aa2e-4a66-ad59-307085eae08c-cert\") pod \"ingress-canary-bzt86\" (UID: \"e5ebef60-aa2e-4a66-ad59-307085eae08c\") " pod="openshift-ingress-canary/ingress-canary-bzt86"
May 29 02:53:05.112894 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:05.112874    2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 02:53:05.112946 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:05.112935    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5ebef60-aa2e-4a66-ad59-307085eae08c-cert podName:e5ebef60-aa2e-4a66-ad59-307085eae08c nodeName:}" failed. No retries permitted until 2026-05-29 02:53:07.112919775 +0000 UTC m=+37.868234816 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e5ebef60-aa2e-4a66-ad59-307085eae08c-cert") pod "ingress-canary-bzt86" (UID: "e5ebef60-aa2e-4a66-ad59-307085eae08c") : secret "canary-serving-cert" not found
May 29 02:53:06.075771 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:06.075735    2559 generic.go:358] "Generic (PLEG): container finished" podID="cccb57a5-0894-4319-9eeb-2459f94a7a17" containerID="1621f8afb4a0074435ba6dab1a8fdd23af65f1942067f375d0f63892ab28d4ea" exitCode=0
May 29 02:53:06.075771 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:06.075774    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-22clf" event={"ID":"cccb57a5-0894-4319-9eeb-2459f94a7a17","Type":"ContainerDied","Data":"1621f8afb4a0074435ba6dab1a8fdd23af65f1942067f375d0f63892ab28d4ea"}
May 29 02:53:07.025436 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:07.025246    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7cca70d-3eb6-4647-9508-894b981687eb-metrics-tls\") pod \"dns-default-9c6sj\" (UID: \"e7cca70d-3eb6-4647-9508-894b981687eb\") " pod="openshift-dns/dns-default-9c6sj"
May 29 02:53:07.025436 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:07.025387    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 02:53:07.025634 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:07.025495    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7cca70d-3eb6-4647-9508-894b981687eb-metrics-tls podName:e7cca70d-3eb6-4647-9508-894b981687eb nodeName:}" failed. No retries permitted until 2026-05-29 02:53:11.025479806 +0000 UTC m=+41.780794848 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7cca70d-3eb6-4647-9508-894b981687eb-metrics-tls") pod "dns-default-9c6sj" (UID: "e7cca70d-3eb6-4647-9508-894b981687eb") : secret "dns-default-metrics-tls" not found
May 29 02:53:07.080645 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:07.080613    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-22clf" event={"ID":"cccb57a5-0894-4319-9eeb-2459f94a7a17","Type":"ContainerStarted","Data":"c774e4ef3dda568fdd6a95f47e7af8c240cd737279678a6b240820408c3fbcd7"}
May 29 02:53:07.103159 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:07.103112    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-22clf" podStartSLOduration=4.288051626 podStartE2EDuration="37.10310062s" podCreationTimestamp="2026-05-29 02:52:30 +0000 UTC" firstStartedPulling="2026-05-29 02:52:31.12101254 +0000 UTC m=+1.876327582" lastFinishedPulling="2026-05-29 02:53:03.936061527 +0000 UTC m=+34.691376576" observedRunningTime="2026-05-29 02:53:07.102822917 +0000 UTC m=+37.858137982" watchObservedRunningTime="2026-05-29 02:53:07.10310062 +0000 UTC m=+37.858415711"
May 29 02:53:07.126190 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:07.126163    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5ebef60-aa2e-4a66-ad59-307085eae08c-cert\") pod \"ingress-canary-bzt86\" (UID: \"e5ebef60-aa2e-4a66-ad59-307085eae08c\") " pod="openshift-ingress-canary/ingress-canary-bzt86"
May 29 02:53:07.126301 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:07.126284    2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 02:53:07.126346 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:07.126337    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5ebef60-aa2e-4a66-ad59-307085eae08c-cert podName:e5ebef60-aa2e-4a66-ad59-307085eae08c nodeName:}" failed. No retries permitted until 2026-05-29 02:53:11.126324811 +0000 UTC m=+41.881639853 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e5ebef60-aa2e-4a66-ad59-307085eae08c-cert") pod "ingress-canary-bzt86" (UID: "e5ebef60-aa2e-4a66-ad59-307085eae08c") : secret "canary-serving-cert" not found
May 29 02:53:11.051002 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:11.050952    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7cca70d-3eb6-4647-9508-894b981687eb-metrics-tls\") pod \"dns-default-9c6sj\" (UID: \"e7cca70d-3eb6-4647-9508-894b981687eb\") " pod="openshift-dns/dns-default-9c6sj"
May 29 02:53:11.051370 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:11.051090    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 02:53:11.051370 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:11.051153    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7cca70d-3eb6-4647-9508-894b981687eb-metrics-tls podName:e7cca70d-3eb6-4647-9508-894b981687eb nodeName:}" failed. No retries permitted until 2026-05-29 02:53:19.051138071 +0000 UTC m=+49.806453113 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7cca70d-3eb6-4647-9508-894b981687eb-metrics-tls") pod "dns-default-9c6sj" (UID: "e7cca70d-3eb6-4647-9508-894b981687eb") : secret "dns-default-metrics-tls" not found
May 29 02:53:11.151743 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:11.151718    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5ebef60-aa2e-4a66-ad59-307085eae08c-cert\") pod \"ingress-canary-bzt86\" (UID: \"e5ebef60-aa2e-4a66-ad59-307085eae08c\") " pod="openshift-ingress-canary/ingress-canary-bzt86"
May 29 02:53:11.151885 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:11.151847    2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 02:53:11.151921 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:11.151891    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5ebef60-aa2e-4a66-ad59-307085eae08c-cert podName:e5ebef60-aa2e-4a66-ad59-307085eae08c nodeName:}" failed. No retries permitted until 2026-05-29 02:53:19.151878389 +0000 UTC m=+49.907193430 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e5ebef60-aa2e-4a66-ad59-307085eae08c-cert") pod "ingress-canary-bzt86" (UID: "e5ebef60-aa2e-4a66-ad59-307085eae08c") : secret "canary-serving-cert" not found
May 29 02:53:19.099932 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:19.099895    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7cca70d-3eb6-4647-9508-894b981687eb-metrics-tls\") pod \"dns-default-9c6sj\" (UID: \"e7cca70d-3eb6-4647-9508-894b981687eb\") " pod="openshift-dns/dns-default-9c6sj"
May 29 02:53:19.100349 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:19.100033    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 02:53:19.100349 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:19.100097    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7cca70d-3eb6-4647-9508-894b981687eb-metrics-tls podName:e7cca70d-3eb6-4647-9508-894b981687eb nodeName:}" failed. No retries permitted until 2026-05-29 02:53:35.100079534 +0000 UTC m=+65.855394579 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7cca70d-3eb6-4647-9508-894b981687eb-metrics-tls") pod "dns-default-9c6sj" (UID: "e7cca70d-3eb6-4647-9508-894b981687eb") : secret "dns-default-metrics-tls" not found
May 29 02:53:19.200665 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:19.200635    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5ebef60-aa2e-4a66-ad59-307085eae08c-cert\") pod \"ingress-canary-bzt86\" (UID: \"e5ebef60-aa2e-4a66-ad59-307085eae08c\") " pod="openshift-ingress-canary/ingress-canary-bzt86"
May 29 02:53:19.200797 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:19.200770    2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 02:53:19.200845 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:19.200835    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5ebef60-aa2e-4a66-ad59-307085eae08c-cert podName:e5ebef60-aa2e-4a66-ad59-307085eae08c nodeName:}" failed. No retries permitted until 2026-05-29 02:53:35.200820671 +0000 UTC m=+65.956135712 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e5ebef60-aa2e-4a66-ad59-307085eae08c-cert") pod "ingress-canary-bzt86" (UID: "e5ebef60-aa2e-4a66-ad59-307085eae08c") : secret "canary-serving-cert" not found
May 29 02:53:30.067120 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.067091    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-psj9n"
May 29 02:53:30.099821 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.099793    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-x2nj4"]
May 29 02:53:30.102406 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.102340    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mtt7h"]
May 29 02:53:30.102523 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.102495    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-x2nj4"
May 29 02:53:30.104228 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.104209    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mtt7h"
May 29 02:53:30.105675 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.105246    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
May 29 02:53:30.105675 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.105349    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
May 29 02:53:30.105675 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.105616    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-fvwkf\""
May 29 02:53:30.105882 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.105848    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
May 29 02:53:30.106464 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.106444    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-pl94j\""
May 29 02:53:30.106642 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.106618    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
May 29 02:53:30.106836 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.106818    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
May 29 02:53:30.110842 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.110766    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mtt7h"]
May 29 02:53:30.111599 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.111578    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-x2nj4"]
May 29 02:53:30.274239 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.274205    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncml8\" (UniqueName: \"kubernetes.io/projected/b13aaac1-8719-4128-ad12-9e0463049b28-kube-api-access-ncml8\") pod \"cluster-samples-operator-697587599-x2nj4\" (UID: \"b13aaac1-8719-4128-ad12-9e0463049b28\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-x2nj4"
May 29 02:53:30.274393 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.274267    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b13aaac1-8719-4128-ad12-9e0463049b28-samples-operator-tls\") pod \"cluster-samples-operator-697587599-x2nj4\" (UID: \"b13aaac1-8719-4128-ad12-9e0463049b28\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-x2nj4"
May 29 02:53:30.274393 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.274301    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptn7f\" (UniqueName: \"kubernetes.io/projected/300ba3a2-0ab1-4944-b6f6-03922b888d1b-kube-api-access-ptn7f\") pod \"volume-data-source-validator-5cf8c89d64-mtt7h\" (UID: \"300ba3a2-0ab1-4944-b6f6-03922b888d1b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mtt7h"
May 29 02:53:30.296801 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.296775    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-z2zvs"]
May 29 02:53:30.298743 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.298727    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-z2zvs"
May 29 02:53:30.300688 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.300666    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
May 29 02:53:30.300688 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.300666    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
May 29 02:53:30.300943 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.300720    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
May 29 02:53:30.300943 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.300674    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
May 29 02:53:30.300943 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.300843    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-wn9zn\""
May 29 02:53:30.308437 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.308417    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-z2zvs"]
May 29 02:53:30.375196 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.375174    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ncml8\" (UniqueName: \"kubernetes.io/projected/b13aaac1-8719-4128-ad12-9e0463049b28-kube-api-access-ncml8\") pod \"cluster-samples-operator-697587599-x2nj4\" (UID: \"b13aaac1-8719-4128-ad12-9e0463049b28\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-x2nj4"
May 29 02:53:30.375292 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.375220    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b13aaac1-8719-4128-ad12-9e0463049b28-samples-operator-tls\") pod \"cluster-samples-operator-697587599-x2nj4\" (UID: \"b13aaac1-8719-4128-ad12-9e0463049b28\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-x2nj4"
May 29 02:53:30.375292 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.375237    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ptn7f\" (UniqueName: \"kubernetes.io/projected/300ba3a2-0ab1-4944-b6f6-03922b888d1b-kube-api-access-ptn7f\") pod \"volume-data-source-validator-5cf8c89d64-mtt7h\" (UID: \"300ba3a2-0ab1-4944-b6f6-03922b888d1b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mtt7h"
May 29 02:53:30.375374 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:30.375318    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 29 02:53:30.375413 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:30.375381    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b13aaac1-8719-4128-ad12-9e0463049b28-samples-operator-tls podName:b13aaac1-8719-4128-ad12-9e0463049b28 nodeName:}" failed. No retries permitted until 2026-05-29 02:53:30.875364267 +0000 UTC m=+61.630679309 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b13aaac1-8719-4128-ad12-9e0463049b28-samples-operator-tls") pod "cluster-samples-operator-697587599-x2nj4" (UID: "b13aaac1-8719-4128-ad12-9e0463049b28") : secret "samples-operator-tls" not found
May 29 02:53:30.385766 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.385748    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptn7f\" (UniqueName: \"kubernetes.io/projected/300ba3a2-0ab1-4944-b6f6-03922b888d1b-kube-api-access-ptn7f\") pod \"volume-data-source-validator-5cf8c89d64-mtt7h\" (UID: \"300ba3a2-0ab1-4944-b6f6-03922b888d1b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mtt7h"
May 29 02:53:30.385976 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.385946    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncml8\" (UniqueName: \"kubernetes.io/projected/b13aaac1-8719-4128-ad12-9e0463049b28-kube-api-access-ncml8\") pod \"cluster-samples-operator-697587599-x2nj4\" (UID: \"b13aaac1-8719-4128-ad12-9e0463049b28\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-x2nj4"
May 29 02:53:30.420837 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.420818    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mtt7h"
May 29 02:53:30.475717 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.475679    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d897b3db-7602-4711-96e7-b21b92e4b006-config\") pod \"service-ca-operator-57b5b68b8-z2zvs\" (UID: \"d897b3db-7602-4711-96e7-b21b92e4b006\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-z2zvs"
May 29 02:53:30.475838 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.475751    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghsf2\" (UniqueName: \"kubernetes.io/projected/d897b3db-7602-4711-96e7-b21b92e4b006-kube-api-access-ghsf2\") pod \"service-ca-operator-57b5b68b8-z2zvs\" (UID: \"d897b3db-7602-4711-96e7-b21b92e4b006\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-z2zvs"
May 29 02:53:30.475916 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.475882    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d897b3db-7602-4711-96e7-b21b92e4b006-serving-cert\") pod \"service-ca-operator-57b5b68b8-z2zvs\" (UID: \"d897b3db-7602-4711-96e7-b21b92e4b006\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-z2zvs"
May 29 02:53:30.556517 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.556488    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mtt7h"]
May 29 02:53:30.560376 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:53:30.560349    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod300ba3a2_0ab1_4944_b6f6_03922b888d1b.slice/crio-f7bc05b13d95f832b42ac33f483f932449ab568b867f715da006073bb39ba322 WatchSource:0}: Error finding container f7bc05b13d95f832b42ac33f483f932449ab568b867f715da006073bb39ba322: Status 404 returned error can't find the container with id f7bc05b13d95f832b42ac33f483f932449ab568b867f715da006073bb39ba322
May 29 02:53:30.576664 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.576639    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d897b3db-7602-4711-96e7-b21b92e4b006-config\") pod \"service-ca-operator-57b5b68b8-z2zvs\" (UID: \"d897b3db-7602-4711-96e7-b21b92e4b006\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-z2zvs"
May 29 02:53:30.576748 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.576684    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ghsf2\" (UniqueName: \"kubernetes.io/projected/d897b3db-7602-4711-96e7-b21b92e4b006-kube-api-access-ghsf2\") pod \"service-ca-operator-57b5b68b8-z2zvs\" (UID: \"d897b3db-7602-4711-96e7-b21b92e4b006\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-z2zvs"
May 29 02:53:30.576748 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.576744    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d897b3db-7602-4711-96e7-b21b92e4b006-serving-cert\") pod \"service-ca-operator-57b5b68b8-z2zvs\" (UID: \"d897b3db-7602-4711-96e7-b21b92e4b006\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-z2zvs"
May 29 02:53:30.577149 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.577129    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d897b3db-7602-4711-96e7-b21b92e4b006-config\") pod \"service-ca-operator-57b5b68b8-z2zvs\" (UID: \"d897b3db-7602-4711-96e7-b21b92e4b006\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-z2zvs"
May 29 02:53:30.578791 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.578772    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d897b3db-7602-4711-96e7-b21b92e4b006-serving-cert\") pod \"service-ca-operator-57b5b68b8-z2zvs\" (UID: \"d897b3db-7602-4711-96e7-b21b92e4b006\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-z2zvs"
May 29 02:53:30.584487 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.584461    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghsf2\" (UniqueName: \"kubernetes.io/projected/d897b3db-7602-4711-96e7-b21b92e4b006-kube-api-access-ghsf2\") pod \"service-ca-operator-57b5b68b8-z2zvs\" (UID: \"d897b3db-7602-4711-96e7-b21b92e4b006\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-z2zvs"
May 29 02:53:30.607559 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.607539    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-z2zvs"
May 29 02:53:30.716450 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.716420    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-z2zvs"]
May 29 02:53:30.719310 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:53:30.719284    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd897b3db_7602_4711_96e7_b21b92e4b006.slice/crio-5f8f3000c99f84eda2f4bb0c129086a0cfe020c089c71d817506b1f868bda2f3 WatchSource:0}: Error finding container 5f8f3000c99f84eda2f4bb0c129086a0cfe020c089c71d817506b1f868bda2f3: Status 404 returned error can't find the container with id 5f8f3000c99f84eda2f4bb0c129086a0cfe020c089c71d817506b1f868bda2f3
May 29 02:53:30.879363 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:30.879330    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b13aaac1-8719-4128-ad12-9e0463049b28-samples-operator-tls\") pod \"cluster-samples-operator-697587599-x2nj4\" (UID: \"b13aaac1-8719-4128-ad12-9e0463049b28\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-x2nj4"
May 29 02:53:30.879507 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:30.879468    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 29 02:53:30.879547 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:30.879527    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b13aaac1-8719-4128-ad12-9e0463049b28-samples-operator-tls podName:b13aaac1-8719-4128-ad12-9e0463049b28 nodeName:}" failed. No retries permitted until 2026-05-29 02:53:31.879510878 +0000 UTC m=+62.634825920 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b13aaac1-8719-4128-ad12-9e0463049b28-samples-operator-tls") pod "cluster-samples-operator-697587599-x2nj4" (UID: "b13aaac1-8719-4128-ad12-9e0463049b28") : secret "samples-operator-tls" not found
May 29 02:53:31.123654 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:31.123618    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-z2zvs" event={"ID":"d897b3db-7602-4711-96e7-b21b92e4b006","Type":"ContainerStarted","Data":"5f8f3000c99f84eda2f4bb0c129086a0cfe020c089c71d817506b1f868bda2f3"}
May 29 02:53:31.124641 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:31.124620    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mtt7h" event={"ID":"300ba3a2-0ab1-4944-b6f6-03922b888d1b","Type":"ContainerStarted","Data":"f7bc05b13d95f832b42ac33f483f932449ab568b867f715da006073bb39ba322"}
May 29 02:53:31.886601 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:31.886565    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b13aaac1-8719-4128-ad12-9e0463049b28-samples-operator-tls\") pod \"cluster-samples-operator-697587599-x2nj4\" (UID: \"b13aaac1-8719-4128-ad12-9e0463049b28\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-x2nj4"
May 29 02:53:31.886774 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:31.886705    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 29 02:53:31.886774 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:31.886770    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b13aaac1-8719-4128-ad12-9e0463049b28-samples-operator-tls podName:b13aaac1-8719-4128-ad12-9e0463049b28 nodeName:}" failed. No retries permitted until 2026-05-29 02:53:33.886753319 +0000 UTC m=+64.642068361 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b13aaac1-8719-4128-ad12-9e0463049b28-samples-operator-tls") pod "cluster-samples-operator-697587599-x2nj4" (UID: "b13aaac1-8719-4128-ad12-9e0463049b28") : secret "samples-operator-tls" not found
May 29 02:53:33.129562 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:33.129520    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-z2zvs" event={"ID":"d897b3db-7602-4711-96e7-b21b92e4b006","Type":"ContainerStarted","Data":"09f05cd6235f03d9f39a5aa80f5593c3819963dde3a9f9f94153dd826518cf5b"}
May 29 02:53:33.130736 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:33.130710    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mtt7h" event={"ID":"300ba3a2-0ab1-4944-b6f6-03922b888d1b","Type":"ContainerStarted","Data":"b29dbcdb35c60b224e231710d0d9a9e68d5772e80b877dcb01b2c928d77bf330"}
May 29 02:53:33.144980 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:33.144918    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-z2zvs" podStartSLOduration=1.087833861 podStartE2EDuration="3.144907384s" podCreationTimestamp="2026-05-29 02:53:30 +0000 UTC" firstStartedPulling="2026-05-29 02:53:30.721158834 +0000 UTC m=+61.476473876" lastFinishedPulling="2026-05-29 02:53:32.778232357 +0000 UTC m=+63.533547399" observedRunningTime="2026-05-29 02:53:33.144284461 +0000 UTC m=+63.899599527" watchObservedRunningTime="2026-05-29 02:53:33.144907384 +0000 UTC m=+63.900222447"
May 29 02:53:33.157580 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:33.157544    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mtt7h" podStartSLOduration=1.50604421 podStartE2EDuration="3.157531708s" podCreationTimestamp="2026-05-29 02:53:30 +0000 UTC" firstStartedPulling="2026-05-29 02:53:30.562123048 +0000 UTC m=+61.317438090" lastFinishedPulling="2026-05-29 02:53:32.213610547 +0000 UTC m=+62.968925588" observedRunningTime="2026-05-29 02:53:33.156783077 +0000 UTC m=+63.912098140" watchObservedRunningTime="2026-05-29 02:53:33.157531708 +0000 UTC m=+63.912846771"
May 29 02:53:33.901958 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:33.901924    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b13aaac1-8719-4128-ad12-9e0463049b28-samples-operator-tls\") pod \"cluster-samples-operator-697587599-x2nj4\" (UID: \"b13aaac1-8719-4128-ad12-9e0463049b28\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-x2nj4"
May 29 02:53:33.902117 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:33.902062    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 29 02:53:33.902156 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:33.902121    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b13aaac1-8719-4128-ad12-9e0463049b28-samples-operator-tls podName:b13aaac1-8719-4128-ad12-9e0463049b28 nodeName:}" failed. No retries permitted until 2026-05-29 02:53:37.902105852 +0000 UTC m=+68.657420899 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b13aaac1-8719-4128-ad12-9e0463049b28-samples-operator-tls") pod "cluster-samples-operator-697587599-x2nj4" (UID: "b13aaac1-8719-4128-ad12-9e0463049b28") : secret "samples-operator-tls" not found
May 29 02:53:34.506592 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:34.506555    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs\") pod \"network-metrics-daemon-r4cd8\" (UID: \"757d1101-4ef1-451f-b984-012545775d40\") " pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:53:34.508417 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:34.508400    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 29 02:53:34.516759 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:34.516734    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 29 02:53:34.516831 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:34.516796    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs podName:757d1101-4ef1-451f-b984-012545775d40 nodeName:}" failed. No retries permitted until 2026-05-29 02:54:38.516780951 +0000 UTC m=+129.272095997 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs") pod "network-metrics-daemon-r4cd8" (UID: "757d1101-4ef1-451f-b984-012545775d40") : secret "metrics-daemon-secret" not found
May 29 02:53:34.708668 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:34.708637    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zw6qf\" (UniqueName: \"kubernetes.io/projected/902df08f-86be-4b22-8929-a7087c7b2c9a-kube-api-access-zw6qf\") pod \"network-check-target-fnzpl\" (UID: \"902df08f-86be-4b22-8929-a7087c7b2c9a\") " pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:53:34.710467 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:34.710451    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 29 02:53:34.720837 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:34.720817    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 29 02:53:34.731539 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:34.731520    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zw6qf\" (UniqueName: \"kubernetes.io/projected/902df08f-86be-4b22-8929-a7087c7b2c9a-kube-api-access-zw6qf\") pod \"network-check-target-fnzpl\" (UID: \"902df08f-86be-4b22-8929-a7087c7b2c9a\") " pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:53:34.859145 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:34.859123    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-r7pnj\""
May 29 02:53:34.867694 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:34.867674    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:53:34.983698 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:34.983671    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fnzpl"]
May 29 02:53:34.988084 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:53:34.988061    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod902df08f_86be_4b22_8929_a7087c7b2c9a.slice/crio-7e508fe4e992836c500589af166b75f3f51e7dc33a516c31c222fe7976d07510 WatchSource:0}: Error finding container 7e508fe4e992836c500589af166b75f3f51e7dc33a516c31c222fe7976d07510: Status 404 returned error can't find the container with id 7e508fe4e992836c500589af166b75f3f51e7dc33a516c31c222fe7976d07510
May 29 02:53:35.109873 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:35.109805    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7cca70d-3eb6-4647-9508-894b981687eb-metrics-tls\") pod \"dns-default-9c6sj\" (UID: \"e7cca70d-3eb6-4647-9508-894b981687eb\") " pod="openshift-dns/dns-default-9c6sj"
May 29 02:53:35.109992 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:35.109942    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 02:53:35.110036 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:35.110016    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7cca70d-3eb6-4647-9508-894b981687eb-metrics-tls podName:e7cca70d-3eb6-4647-9508-894b981687eb nodeName:}" failed. No retries permitted until 2026-05-29 02:54:07.110001933 +0000 UTC m=+97.865316975 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7cca70d-3eb6-4647-9508-894b981687eb-metrics-tls") pod "dns-default-9c6sj" (UID: "e7cca70d-3eb6-4647-9508-894b981687eb") : secret "dns-default-metrics-tls" not found
May 29 02:53:35.135775 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:35.135742    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fnzpl" event={"ID":"902df08f-86be-4b22-8929-a7087c7b2c9a","Type":"ContainerStarted","Data":"7e508fe4e992836c500589af166b75f3f51e7dc33a516c31c222fe7976d07510"}
May 29 02:53:35.210695 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:35.210667    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5ebef60-aa2e-4a66-ad59-307085eae08c-cert\") pod \"ingress-canary-bzt86\" (UID: \"e5ebef60-aa2e-4a66-ad59-307085eae08c\") " pod="openshift-ingress-canary/ingress-canary-bzt86"
May 29 02:53:35.210803 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:35.210791    2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 02:53:35.210850 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:35.210842    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5ebef60-aa2e-4a66-ad59-307085eae08c-cert podName:e5ebef60-aa2e-4a66-ad59-307085eae08c nodeName:}" failed. No retries permitted until 2026-05-29 02:54:07.210827164 +0000 UTC m=+97.966142208 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e5ebef60-aa2e-4a66-ad59-307085eae08c-cert") pod "ingress-canary-bzt86" (UID: "e5ebef60-aa2e-4a66-ad59-307085eae08c") : secret "canary-serving-cert" not found
May 29 02:53:35.227662 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:35.227636    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-xgkbz"]
May 29 02:53:35.230213 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:35.230198    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-xgkbz"
May 29 02:53:35.232425 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:35.232406    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-f97kf\""
May 29 02:53:35.232508 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:35.232410    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
May 29 02:53:35.232508 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:35.232500    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
May 29 02:53:35.239462 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:35.239441    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-xgkbz"]
May 29 02:53:35.411439 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:35.411372    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g7nz\" (UniqueName: \"kubernetes.io/projected/a79bccd9-301d-4bfe-add6-f2a150ae2366-kube-api-access-8g7nz\") pod \"migrator-b985f5d89-xgkbz\" (UID: \"a79bccd9-301d-4bfe-add6-f2a150ae2366\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-xgkbz"
May 29 02:53:35.512548 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:35.512506    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8g7nz\" (UniqueName: \"kubernetes.io/projected/a79bccd9-301d-4bfe-add6-f2a150ae2366-kube-api-access-8g7nz\") pod \"migrator-b985f5d89-xgkbz\" (UID: \"a79bccd9-301d-4bfe-add6-f2a150ae2366\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-xgkbz"
May 29 02:53:35.530677 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:35.530648    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g7nz\" (UniqueName: \"kubernetes.io/projected/a79bccd9-301d-4bfe-add6-f2a150ae2366-kube-api-access-8g7nz\") pod \"migrator-b985f5d89-xgkbz\" (UID: \"a79bccd9-301d-4bfe-add6-f2a150ae2366\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-xgkbz"
May 29 02:53:35.538453 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:35.538431    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-xgkbz"
May 29 02:53:35.676363 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:35.676291    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-xgkbz"]
May 29 02:53:35.680463 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:53:35.680433    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda79bccd9_301d_4bfe_add6_f2a150ae2366.slice/crio-40d425add6aa389395f6ed111fe456ab7f4d07f82f926d85a7074942e149e8eb WatchSource:0}: Error finding container 40d425add6aa389395f6ed111fe456ab7f4d07f82f926d85a7074942e149e8eb: Status 404 returned error can't find the container with id 40d425add6aa389395f6ed111fe456ab7f4d07f82f926d85a7074942e149e8eb
May 29 02:53:35.991047 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:35.990952    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-lsklf"]
May 29 02:53:35.993695 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:35.993673    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-67d667c7d-lsklf"
May 29 02:53:35.996572 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:35.996158    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-5zhvl\""
May 29 02:53:35.996572 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:35.996188    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
May 29 02:53:35.996572 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:35.996231    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
May 29 02:53:35.996572 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:35.996271    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
May 29 02:53:35.996572 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:35.996158    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
May 29 02:53:36.001054 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:36.001023    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-lsklf"]
May 29 02:53:36.117094 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:36.117061    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdlrb\" (UniqueName: \"kubernetes.io/projected/62b9435a-1196-4653-b535-d4ed06119ff6-kube-api-access-qdlrb\") pod \"service-ca-67d667c7d-lsklf\" (UID: \"62b9435a-1196-4653-b535-d4ed06119ff6\") " pod="openshift-service-ca/service-ca-67d667c7d-lsklf"
May 29 02:53:36.117274 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:36.117116    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/62b9435a-1196-4653-b535-d4ed06119ff6-signing-key\") pod \"service-ca-67d667c7d-lsklf\" (UID: \"62b9435a-1196-4653-b535-d4ed06119ff6\") " pod="openshift-service-ca/service-ca-67d667c7d-lsklf"
May 29 02:53:36.117274 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:36.117178    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/62b9435a-1196-4653-b535-d4ed06119ff6-signing-cabundle\") pod \"service-ca-67d667c7d-lsklf\" (UID: \"62b9435a-1196-4653-b535-d4ed06119ff6\") " pod="openshift-service-ca/service-ca-67d667c7d-lsklf"
May 29 02:53:36.138979 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:36.138936    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-xgkbz" event={"ID":"a79bccd9-301d-4bfe-add6-f2a150ae2366","Type":"ContainerStarted","Data":"40d425add6aa389395f6ed111fe456ab7f4d07f82f926d85a7074942e149e8eb"}
May 29 02:53:36.217660 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:36.217610    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/62b9435a-1196-4653-b535-d4ed06119ff6-signing-key\") pod \"service-ca-67d667c7d-lsklf\" (UID: \"62b9435a-1196-4653-b535-d4ed06119ff6\") " pod="openshift-service-ca/service-ca-67d667c7d-lsklf"
May 29 02:53:36.217824 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:36.217753    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/62b9435a-1196-4653-b535-d4ed06119ff6-signing-cabundle\") pod \"service-ca-67d667c7d-lsklf\" (UID: \"62b9435a-1196-4653-b535-d4ed06119ff6\") " pod="openshift-service-ca/service-ca-67d667c7d-lsklf"
May 29 02:53:36.217894 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:36.217870    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qdlrb\" (UniqueName: \"kubernetes.io/projected/62b9435a-1196-4653-b535-d4ed06119ff6-kube-api-access-qdlrb\") pod \"service-ca-67d667c7d-lsklf\" (UID: \"62b9435a-1196-4653-b535-d4ed06119ff6\") " pod="openshift-service-ca/service-ca-67d667c7d-lsklf"
May 29 02:53:36.218770 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:36.218742    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/62b9435a-1196-4653-b535-d4ed06119ff6-signing-cabundle\") pod \"service-ca-67d667c7d-lsklf\" (UID: \"62b9435a-1196-4653-b535-d4ed06119ff6\") " pod="openshift-service-ca/service-ca-67d667c7d-lsklf"
May 29 02:53:36.220493 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:36.220467    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/62b9435a-1196-4653-b535-d4ed06119ff6-signing-key\") pod \"service-ca-67d667c7d-lsklf\" (UID: \"62b9435a-1196-4653-b535-d4ed06119ff6\") " pod="openshift-service-ca/service-ca-67d667c7d-lsklf"
May 29 02:53:36.226293 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:36.226256    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdlrb\" (UniqueName: \"kubernetes.io/projected/62b9435a-1196-4653-b535-d4ed06119ff6-kube-api-access-qdlrb\") pod \"service-ca-67d667c7d-lsklf\" (UID: \"62b9435a-1196-4653-b535-d4ed06119ff6\") " pod="openshift-service-ca/service-ca-67d667c7d-lsklf"
May 29 02:53:36.307753 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:36.307675    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-67d667c7d-lsklf"
May 29 02:53:36.445178 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:36.445126    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-lsklf"]
May 29 02:53:36.450577 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:53:36.450548    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62b9435a_1196_4653_b535_d4ed06119ff6.slice/crio-e1a93ebdc5b6db6c59ec0382fec011250418c6c24e5afc01b6f1c4103987d977 WatchSource:0}: Error finding container e1a93ebdc5b6db6c59ec0382fec011250418c6c24e5afc01b6f1c4103987d977: Status 404 returned error can't find the container with id e1a93ebdc5b6db6c59ec0382fec011250418c6c24e5afc01b6f1c4103987d977
May 29 02:53:37.142811 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:37.142770    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-67d667c7d-lsklf" event={"ID":"62b9435a-1196-4653-b535-d4ed06119ff6","Type":"ContainerStarted","Data":"fe42491e144dc91acdeffc6948d8e7d03ce3a1ae0cb70fcca7661330a5a07b0c"}
May 29 02:53:37.142811 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:37.142812    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-67d667c7d-lsklf" event={"ID":"62b9435a-1196-4653-b535-d4ed06119ff6","Type":"ContainerStarted","Data":"e1a93ebdc5b6db6c59ec0382fec011250418c6c24e5afc01b6f1c4103987d977"}
May 29 02:53:37.158127 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:37.158076    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-67d667c7d-lsklf" podStartSLOduration=2.158061477 podStartE2EDuration="2.158061477s" podCreationTimestamp="2026-05-29 02:53:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 02:53:37.15781559 +0000 UTC m=+67.913130654" watchObservedRunningTime="2026-05-29 02:53:37.158061477 +0000 UTC m=+67.913376538"
May 29 02:53:37.933111 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:37.932524    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b13aaac1-8719-4128-ad12-9e0463049b28-samples-operator-tls\") pod \"cluster-samples-operator-697587599-x2nj4\" (UID: \"b13aaac1-8719-4128-ad12-9e0463049b28\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-x2nj4"
May 29 02:53:37.933111 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:37.932668    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 29 02:53:37.933111 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:37.932729    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b13aaac1-8719-4128-ad12-9e0463049b28-samples-operator-tls podName:b13aaac1-8719-4128-ad12-9e0463049b28 nodeName:}" failed. No retries permitted until 2026-05-29 02:53:45.932710521 +0000 UTC m=+76.688025566 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b13aaac1-8719-4128-ad12-9e0463049b28-samples-operator-tls") pod "cluster-samples-operator-697587599-x2nj4" (UID: "b13aaac1-8719-4128-ad12-9e0463049b28") : secret "samples-operator-tls" not found
May 29 02:53:38.610289 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:38.610263    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wz44t_846260de-5ed8-4c80-ad40-8480a09eddd2/dns-node-resolver/0.log"
May 29 02:53:39.149895 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:39.149800    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fnzpl" event={"ID":"902df08f-86be-4b22-8929-a7087c7b2c9a","Type":"ContainerStarted","Data":"6a3dea52e7fb95cc6db62be667f3be1ce00beadf24a28adbda15664accf1667d"}
May 29 02:53:39.150096 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:39.149904    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:53:39.151287 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:39.151264    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-xgkbz" event={"ID":"a79bccd9-301d-4bfe-add6-f2a150ae2366","Type":"ContainerStarted","Data":"0c1a972f966b0490a71e4f55d9b32c74e1d21a42c22e57c404808b476c1406d4"}
May 29 02:53:39.151381 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:39.151290    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-xgkbz" event={"ID":"a79bccd9-301d-4bfe-add6-f2a150ae2366","Type":"ContainerStarted","Data":"14a428712f80f43b891d3091a8c9603bcfd22580ae2cb03327075ea7275f01e1"}
May 29 02:53:39.165898 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:39.165846    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fnzpl" podStartSLOduration=65.407433585 podStartE2EDuration="1m9.165829905s" podCreationTimestamp="2026-05-29 02:52:30 +0000 UTC" firstStartedPulling="2026-05-29 02:53:34.990417651 +0000 UTC m=+65.745732698" lastFinishedPulling="2026-05-29 02:53:38.748813977 +0000 UTC m=+69.504129018" observedRunningTime="2026-05-29 02:53:39.164040549 +0000 UTC m=+69.919355614" watchObservedRunningTime="2026-05-29 02:53:39.165829905 +0000 UTC m=+69.921144970"
May 29 02:53:39.177698 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:39.177645    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-xgkbz" podStartSLOduration=1.120999583 podStartE2EDuration="4.17763584s" podCreationTimestamp="2026-05-29 02:53:35 +0000 UTC" firstStartedPulling="2026-05-29 02:53:35.682549776 +0000 UTC m=+66.437864818" lastFinishedPulling="2026-05-29 02:53:38.739186034 +0000 UTC m=+69.494501075" observedRunningTime="2026-05-29 02:53:39.17749927 +0000 UTC m=+69.932814333" watchObservedRunningTime="2026-05-29 02:53:39.17763584 +0000 UTC m=+69.932950904"
May 29 02:53:39.210256 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:39.210232    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-92nfp_235d8ad4-77e9-477a-b8d4-4edc46151e4c/node-ca/0.log"
May 29 02:53:40.610520 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:40.610494    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-xgkbz_a79bccd9-301d-4bfe-add6-f2a150ae2366/migrator/0.log"
May 29 02:53:40.809841 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:40.809806    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-xgkbz_a79bccd9-301d-4bfe-add6-f2a150ae2366/graceful-termination/0.log"
May 29 02:53:45.992204 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:45.992151    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b13aaac1-8719-4128-ad12-9e0463049b28-samples-operator-tls\") pod \"cluster-samples-operator-697587599-x2nj4\" (UID: \"b13aaac1-8719-4128-ad12-9e0463049b28\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-x2nj4"
May 29 02:53:45.994605 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:45.994580    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b13aaac1-8719-4128-ad12-9e0463049b28-samples-operator-tls\") pod \"cluster-samples-operator-697587599-x2nj4\" (UID: \"b13aaac1-8719-4128-ad12-9e0463049b28\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-x2nj4"
May 29 02:53:46.014602 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:46.014575    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-x2nj4"
May 29 02:53:46.138139 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:46.138107    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-x2nj4"]
May 29 02:53:47.172724 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:47.172684    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-x2nj4" event={"ID":"b13aaac1-8719-4128-ad12-9e0463049b28","Type":"ContainerStarted","Data":"37c3f4592fa8e2941151dd452045e57bb3b55e48d419c021b48a913d86320bdd"}
May 29 02:53:48.179799 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:48.179763    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-x2nj4" event={"ID":"b13aaac1-8719-4128-ad12-9e0463049b28","Type":"ContainerStarted","Data":"a1501ee0c0f915d16e675ab04d4389f8860944ad960c37221925dae2cda3d173"}
May 29 02:53:48.179799 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:48.179800    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-x2nj4" event={"ID":"b13aaac1-8719-4128-ad12-9e0463049b28","Type":"ContainerStarted","Data":"08dd7fe2196cb6925b1d5c679cdb19d756751cf61580a0092c1eb0150c91a5dd"}
May 29 02:53:48.203091 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:48.203046    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-x2nj4" podStartSLOduration=16.397745542 podStartE2EDuration="18.203033003s" podCreationTimestamp="2026-05-29 02:53:30 +0000 UTC" firstStartedPulling="2026-05-29 02:53:46.185470653 +0000 UTC m=+76.940785696" lastFinishedPulling="2026-05-29 02:53:47.990758109 +0000 UTC m=+78.746073157" observedRunningTime="2026-05-29 02:53:48.201710809 +0000 UTC m=+78.957025863" watchObservedRunningTime="2026-05-29 02:53:48.203033003 +0000 UTC m=+78.958348068"
May 29 02:53:57.595672 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.595538    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-54688b74b-pvv7w"]
May 29 02:53:57.599783 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.599761    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:57.601887 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.601867    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 29 02:53:57.602595 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.602574    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 29 02:53:57.602699 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.602579    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5c2jm\""
May 29 02:53:57.602699 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.602583    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 29 02:53:57.608117 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.608097    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 29 02:53:57.612759 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.612740    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54688b74b-pvv7w"]
May 29 02:53:57.620451 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.620423    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-t6s4g"]
May 29 02:53:57.623009 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.622956    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9thz9"]
May 29 02:53:57.623106 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.623055    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-t6s4g"
May 29 02:53:57.624913 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.624897    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
May 29 02:53:57.625396 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.625379    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-4x259\""
May 29 02:53:57.625838 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.625824    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9thz9"
May 29 02:53:57.628112 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.628093    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 29 02:53:57.628218 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.628202    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 29 02:53:57.628466 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.628451    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-n4d48\""
May 29 02:53:57.628538 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.628524    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 29 02:53:57.628784 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.628769    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 29 02:53:57.634911 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.634890    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-t6s4g"]
May 29 02:53:57.637835 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.637816    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9thz9"]
May 29 02:53:57.670994 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.670960    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/76fded48-86f5-4d63-9184-0a8cd59e77ac-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-t6s4g\" (UID: \"76fded48-86f5-4d63-9184-0a8cd59e77ac\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-t6s4g"
May 29 02:53:57.671120 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.671005    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/503df8ce-8b42-4e8e-a59a-92b9a68b979d-data-volume\") pod \"insights-runtime-extractor-9thz9\" (UID: \"503df8ce-8b42-4e8e-a59a-92b9a68b979d\") " pod="openshift-insights/insights-runtime-extractor-9thz9"
May 29 02:53:57.671120 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.671023    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4942eb9f-01a3-4702-a9cc-be97c76ab116-installation-pull-secrets\") pod \"image-registry-54688b74b-pvv7w\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") " pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:57.671120 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.671040    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4942eb9f-01a3-4702-a9cc-be97c76ab116-registry-certificates\") pod \"image-registry-54688b74b-pvv7w\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") " pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:57.671120 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.671076    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4942eb9f-01a3-4702-a9cc-be97c76ab116-trusted-ca\") pod \"image-registry-54688b74b-pvv7w\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") " pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:57.671120 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.671114    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lm9q6\" (UniqueName: \"kubernetes.io/projected/4942eb9f-01a3-4702-a9cc-be97c76ab116-kube-api-access-lm9q6\") pod \"image-registry-54688b74b-pvv7w\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") " pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:57.671378 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.671206    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/503df8ce-8b42-4e8e-a59a-92b9a68b979d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9thz9\" (UID: \"503df8ce-8b42-4e8e-a59a-92b9a68b979d\") " pod="openshift-insights/insights-runtime-extractor-9thz9"
May 29 02:53:57.671378 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.671234    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/503df8ce-8b42-4e8e-a59a-92b9a68b979d-crio-socket\") pod \"insights-runtime-extractor-9thz9\" (UID: \"503df8ce-8b42-4e8e-a59a-92b9a68b979d\") " pod="openshift-insights/insights-runtime-extractor-9thz9"
May 29 02:53:57.671378 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.671266    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4942eb9f-01a3-4702-a9cc-be97c76ab116-bound-sa-token\") pod \"image-registry-54688b74b-pvv7w\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") " pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:57.671378 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.671360    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4942eb9f-01a3-4702-a9cc-be97c76ab116-registry-tls\") pod \"image-registry-54688b74b-pvv7w\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") " pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:57.671564 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.671404    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/503df8ce-8b42-4e8e-a59a-92b9a68b979d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9thz9\" (UID: \"503df8ce-8b42-4e8e-a59a-92b9a68b979d\") " pod="openshift-insights/insights-runtime-extractor-9thz9"
May 29 02:53:57.671564 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.671433    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4942eb9f-01a3-4702-a9cc-be97c76ab116-ca-trust-extracted\") pod \"image-registry-54688b74b-pvv7w\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") " pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:57.671564 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.671466    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvhs6\" (UniqueName: \"kubernetes.io/projected/503df8ce-8b42-4e8e-a59a-92b9a68b979d-kube-api-access-hvhs6\") pod \"insights-runtime-extractor-9thz9\" (UID: \"503df8ce-8b42-4e8e-a59a-92b9a68b979d\") " pod="openshift-insights/insights-runtime-extractor-9thz9"
May 29 02:53:57.671564 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.671492    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4942eb9f-01a3-4702-a9cc-be97c76ab116-image-registry-private-configuration\") pod \"image-registry-54688b74b-pvv7w\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") " pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:57.692745 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.692721    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-54688b74b-pvv7w"]
May 29 02:53:57.692873 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:57.692853    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[bound-sa-token ca-trust-extracted image-registry-private-configuration installation-pull-secrets kube-api-access-lm9q6 registry-certificates registry-tls trusted-ca], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-54688b74b-pvv7w" podUID="4942eb9f-01a3-4702-a9cc-be97c76ab116"
May 29 02:53:57.772070 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.772043    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4942eb9f-01a3-4702-a9cc-be97c76ab116-bound-sa-token\") pod \"image-registry-54688b74b-pvv7w\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") " pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:57.772175 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.772093    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4942eb9f-01a3-4702-a9cc-be97c76ab116-registry-tls\") pod \"image-registry-54688b74b-pvv7w\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") " pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:57.772175 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.772114    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/503df8ce-8b42-4e8e-a59a-92b9a68b979d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9thz9\" (UID: \"503df8ce-8b42-4e8e-a59a-92b9a68b979d\") " pod="openshift-insights/insights-runtime-extractor-9thz9"
May 29 02:53:57.772175 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.772131    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4942eb9f-01a3-4702-a9cc-be97c76ab116-ca-trust-extracted\") pod \"image-registry-54688b74b-pvv7w\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") " pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:57.772175 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.772156    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hvhs6\" (UniqueName: \"kubernetes.io/projected/503df8ce-8b42-4e8e-a59a-92b9a68b979d-kube-api-access-hvhs6\") pod \"insights-runtime-extractor-9thz9\" (UID: \"503df8ce-8b42-4e8e-a59a-92b9a68b979d\") " pod="openshift-insights/insights-runtime-extractor-9thz9"
May 29 02:53:57.772352 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.772284    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4942eb9f-01a3-4702-a9cc-be97c76ab116-image-registry-private-configuration\") pod \"image-registry-54688b74b-pvv7w\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") " pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:57.772407 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.772367    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/76fded48-86f5-4d63-9184-0a8cd59e77ac-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-t6s4g\" (UID: \"76fded48-86f5-4d63-9184-0a8cd59e77ac\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-t6s4g"
May 29 02:53:57.772461 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.772416    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/503df8ce-8b42-4e8e-a59a-92b9a68b979d-data-volume\") pod \"insights-runtime-extractor-9thz9\" (UID: \"503df8ce-8b42-4e8e-a59a-92b9a68b979d\") " pod="openshift-insights/insights-runtime-extractor-9thz9"
May 29 02:53:57.772461 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.772445    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4942eb9f-01a3-4702-a9cc-be97c76ab116-installation-pull-secrets\") pod \"image-registry-54688b74b-pvv7w\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") " pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:57.772564 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.772475    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4942eb9f-01a3-4702-a9cc-be97c76ab116-registry-certificates\") pod \"image-registry-54688b74b-pvv7w\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") " pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:57.772564 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.772503    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4942eb9f-01a3-4702-a9cc-be97c76ab116-trusted-ca\") pod \"image-registry-54688b74b-pvv7w\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") " pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:57.772564 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.772527    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lm9q6\" (UniqueName: \"kubernetes.io/projected/4942eb9f-01a3-4702-a9cc-be97c76ab116-kube-api-access-lm9q6\") pod \"image-registry-54688b74b-pvv7w\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") " pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:57.772714 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.772577    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/503df8ce-8b42-4e8e-a59a-92b9a68b979d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9thz9\" (UID: \"503df8ce-8b42-4e8e-a59a-92b9a68b979d\") " pod="openshift-insights/insights-runtime-extractor-9thz9"
May 29 02:53:57.772714 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.772603    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/503df8ce-8b42-4e8e-a59a-92b9a68b979d-crio-socket\") pod \"insights-runtime-extractor-9thz9\" (UID: \"503df8ce-8b42-4e8e-a59a-92b9a68b979d\") " pod="openshift-insights/insights-runtime-extractor-9thz9"
May 29 02:53:57.772714 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.772611    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4942eb9f-01a3-4702-a9cc-be97c76ab116-ca-trust-extracted\") pod \"image-registry-54688b74b-pvv7w\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") " pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:57.772714 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.772702    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/503df8ce-8b42-4e8e-a59a-92b9a68b979d-crio-socket\") pod \"insights-runtime-extractor-9thz9\" (UID: \"503df8ce-8b42-4e8e-a59a-92b9a68b979d\") " pod="openshift-insights/insights-runtime-extractor-9thz9"
May 29 02:53:57.772916 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.772722    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/503df8ce-8b42-4e8e-a59a-92b9a68b979d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9thz9\" (UID: \"503df8ce-8b42-4e8e-a59a-92b9a68b979d\") " pod="openshift-insights/insights-runtime-extractor-9thz9"
May 29 02:53:57.772916 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.772724    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/503df8ce-8b42-4e8e-a59a-92b9a68b979d-data-volume\") pod \"insights-runtime-extractor-9thz9\" (UID: \"503df8ce-8b42-4e8e-a59a-92b9a68b979d\") " pod="openshift-insights/insights-runtime-extractor-9thz9"
May 29 02:53:57.773506 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.773472    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4942eb9f-01a3-4702-a9cc-be97c76ab116-registry-certificates\") pod \"image-registry-54688b74b-pvv7w\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") " pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:57.773690 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.773666    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4942eb9f-01a3-4702-a9cc-be97c76ab116-trusted-ca\") pod \"image-registry-54688b74b-pvv7w\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") " pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:57.775195 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.775169    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4942eb9f-01a3-4702-a9cc-be97c76ab116-installation-pull-secrets\") pod \"image-registry-54688b74b-pvv7w\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") " pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:57.775277 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.775222    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4942eb9f-01a3-4702-a9cc-be97c76ab116-registry-tls\") pod \"image-registry-54688b74b-pvv7w\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") " pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:57.775277 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.775226    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/76fded48-86f5-4d63-9184-0a8cd59e77ac-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-t6s4g\" (UID: \"76fded48-86f5-4d63-9184-0a8cd59e77ac\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-t6s4g"
May 29 02:53:57.775277 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.775255    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/503df8ce-8b42-4e8e-a59a-92b9a68b979d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9thz9\" (UID: \"503df8ce-8b42-4e8e-a59a-92b9a68b979d\") " pod="openshift-insights/insights-runtime-extractor-9thz9"
May 29 02:53:57.775388 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.775360    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4942eb9f-01a3-4702-a9cc-be97c76ab116-image-registry-private-configuration\") pod \"image-registry-54688b74b-pvv7w\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") " pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:57.784245 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.784227    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvhs6\" (UniqueName: \"kubernetes.io/projected/503df8ce-8b42-4e8e-a59a-92b9a68b979d-kube-api-access-hvhs6\") pod \"insights-runtime-extractor-9thz9\" (UID: \"503df8ce-8b42-4e8e-a59a-92b9a68b979d\") " pod="openshift-insights/insights-runtime-extractor-9thz9"
May 29 02:53:57.784326 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.784276    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4942eb9f-01a3-4702-a9cc-be97c76ab116-bound-sa-token\") pod \"image-registry-54688b74b-pvv7w\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") " pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:57.784487 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.784471    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lm9q6\" (UniqueName: \"kubernetes.io/projected/4942eb9f-01a3-4702-a9cc-be97c76ab116-kube-api-access-lm9q6\") pod \"image-registry-54688b74b-pvv7w\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") " pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:57.933075 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.933049    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-t6s4g"
May 29 02:53:57.937732 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:57.937714    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9thz9"
May 29 02:53:58.083927 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.083898    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-t6s4g"]
May 29 02:53:58.086518 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:53:58.086494    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76fded48_86f5_4d63_9184_0a8cd59e77ac.slice/crio-89f820958636e7bd4e56fa11158dd907809319911dcaece38365ce43c41bd212 WatchSource:0}: Error finding container 89f820958636e7bd4e56fa11158dd907809319911dcaece38365ce43c41bd212: Status 404 returned error can't find the container with id 89f820958636e7bd4e56fa11158dd907809319911dcaece38365ce43c41bd212
May 29 02:53:58.101460 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.101431    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9thz9"]
May 29 02:53:58.104542 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:53:58.104520    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod503df8ce_8b42_4e8e_a59a_92b9a68b979d.slice/crio-b81bff6241d7a612e4019014a24336d75354d968cff0d02d7d2aa593fdc339b0 WatchSource:0}: Error finding container b81bff6241d7a612e4019014a24336d75354d968cff0d02d7d2aa593fdc339b0: Status 404 returned error can't find the container with id b81bff6241d7a612e4019014a24336d75354d968cff0d02d7d2aa593fdc339b0
May 29 02:53:58.202021 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.201920    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-t6s4g" event={"ID":"76fded48-86f5-4d63-9184-0a8cd59e77ac","Type":"ContainerStarted","Data":"89f820958636e7bd4e56fa11158dd907809319911dcaece38365ce43c41bd212"}
May 29 02:53:58.203026 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.202999    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9thz9" event={"ID":"503df8ce-8b42-4e8e-a59a-92b9a68b979d","Type":"ContainerStarted","Data":"20cbe65ae3a07bf3226bcdbcaa801f4c8d385b34094eadb6fdf791ca27ba6556"}
May 29 02:53:58.203123 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.203032    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9thz9" event={"ID":"503df8ce-8b42-4e8e-a59a-92b9a68b979d","Type":"ContainerStarted","Data":"b81bff6241d7a612e4019014a24336d75354d968cff0d02d7d2aa593fdc339b0"}
May 29 02:53:58.203123 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.203054    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:58.206939 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.206924    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:58.276245 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.276220    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4942eb9f-01a3-4702-a9cc-be97c76ab116-installation-pull-secrets\") pod \"4942eb9f-01a3-4702-a9cc-be97c76ab116\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") "
May 29 02:53:58.276348 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.276261    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4942eb9f-01a3-4702-a9cc-be97c76ab116-ca-trust-extracted\") pod \"4942eb9f-01a3-4702-a9cc-be97c76ab116\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") "
May 29 02:53:58.276348 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.276295    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4942eb9f-01a3-4702-a9cc-be97c76ab116-registry-tls\") pod \"4942eb9f-01a3-4702-a9cc-be97c76ab116\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") "
May 29 02:53:58.276348 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.276312    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4942eb9f-01a3-4702-a9cc-be97c76ab116-image-registry-private-configuration\") pod \"4942eb9f-01a3-4702-a9cc-be97c76ab116\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") "
May 29 02:53:58.276480 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.276353    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lm9q6\" (UniqueName: \"kubernetes.io/projected/4942eb9f-01a3-4702-a9cc-be97c76ab116-kube-api-access-lm9q6\") pod \"4942eb9f-01a3-4702-a9cc-be97c76ab116\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") "
May 29 02:53:58.276480 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.276374    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4942eb9f-01a3-4702-a9cc-be97c76ab116-registry-certificates\") pod \"4942eb9f-01a3-4702-a9cc-be97c76ab116\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") "
May 29 02:53:58.276480 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.276403    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4942eb9f-01a3-4702-a9cc-be97c76ab116-trusted-ca\") pod \"4942eb9f-01a3-4702-a9cc-be97c76ab116\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") "
May 29 02:53:58.276480 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.276427    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4942eb9f-01a3-4702-a9cc-be97c76ab116-bound-sa-token\") pod \"4942eb9f-01a3-4702-a9cc-be97c76ab116\" (UID: \"4942eb9f-01a3-4702-a9cc-be97c76ab116\") "
May 29 02:53:58.276700 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.276648    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4942eb9f-01a3-4702-a9cc-be97c76ab116-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4942eb9f-01a3-4702-a9cc-be97c76ab116" (UID: "4942eb9f-01a3-4702-a9cc-be97c76ab116"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 02:53:58.276997 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.276947    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4942eb9f-01a3-4702-a9cc-be97c76ab116-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4942eb9f-01a3-4702-a9cc-be97c76ab116" (UID: "4942eb9f-01a3-4702-a9cc-be97c76ab116"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 02:53:58.277115 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.277001    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4942eb9f-01a3-4702-a9cc-be97c76ab116-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4942eb9f-01a3-4702-a9cc-be97c76ab116" (UID: "4942eb9f-01a3-4702-a9cc-be97c76ab116"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 02:53:58.278601 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.278572    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4942eb9f-01a3-4702-a9cc-be97c76ab116-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4942eb9f-01a3-4702-a9cc-be97c76ab116" (UID: "4942eb9f-01a3-4702-a9cc-be97c76ab116"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 02:53:58.278699 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.278643    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4942eb9f-01a3-4702-a9cc-be97c76ab116-kube-api-access-lm9q6" (OuterVolumeSpecName: "kube-api-access-lm9q6") pod "4942eb9f-01a3-4702-a9cc-be97c76ab116" (UID: "4942eb9f-01a3-4702-a9cc-be97c76ab116"). InnerVolumeSpecName "kube-api-access-lm9q6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 02:53:58.278699 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.278651    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4942eb9f-01a3-4702-a9cc-be97c76ab116-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4942eb9f-01a3-4702-a9cc-be97c76ab116" (UID: "4942eb9f-01a3-4702-a9cc-be97c76ab116"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 02:53:58.278842 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.278824    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4942eb9f-01a3-4702-a9cc-be97c76ab116-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4942eb9f-01a3-4702-a9cc-be97c76ab116" (UID: "4942eb9f-01a3-4702-a9cc-be97c76ab116"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:53:58.278878 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.278842    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4942eb9f-01a3-4702-a9cc-be97c76ab116-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4942eb9f-01a3-4702-a9cc-be97c76ab116" (UID: "4942eb9f-01a3-4702-a9cc-be97c76ab116"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:53:58.377364 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.377340    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lm9q6\" (UniqueName: \"kubernetes.io/projected/4942eb9f-01a3-4702-a9cc-be97c76ab116-kube-api-access-lm9q6\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:53:58.377364 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.377360    2559 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4942eb9f-01a3-4702-a9cc-be97c76ab116-registry-certificates\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:53:58.377493 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.377371    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4942eb9f-01a3-4702-a9cc-be97c76ab116-trusted-ca\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:53:58.377493 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.377382    2559 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4942eb9f-01a3-4702-a9cc-be97c76ab116-bound-sa-token\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:53:58.377493 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.377391    2559 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4942eb9f-01a3-4702-a9cc-be97c76ab116-installation-pull-secrets\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:53:58.377493 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.377399    2559 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4942eb9f-01a3-4702-a9cc-be97c76ab116-ca-trust-extracted\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:53:58.377493 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.377407    2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4942eb9f-01a3-4702-a9cc-be97c76ab116-registry-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:53:58.377493 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:58.377416    2559 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4942eb9f-01a3-4702-a9cc-be97c76ab116-image-registry-private-configuration\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:53:59.206652 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.206616    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-t6s4g" event={"ID":"76fded48-86f5-4d63-9184-0a8cd59e77ac","Type":"ContainerStarted","Data":"842a04292fb5ed76b07bb4476b993b3c991edcc94ca2832260da55f7d79c795c"}
May 29 02:53:59.207182 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.206796    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-t6s4g"
May 29 02:53:59.208228 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.208202    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9thz9" event={"ID":"503df8ce-8b42-4e8e-a59a-92b9a68b979d","Type":"ContainerStarted","Data":"1865fcedbb9556de942e8b6b8e70bf50757ea5cf50d14e412e5e41aae7271aa3"}
May 29 02:53:59.208228 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.208228    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54688b74b-pvv7w"
May 29 02:53:59.212983 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.212942    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-t6s4g"
May 29 02:53:59.223094 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.223050    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-t6s4g" podStartSLOduration=1.193673309 podStartE2EDuration="2.223034876s" podCreationTimestamp="2026-05-29 02:53:57 +0000 UTC" firstStartedPulling="2026-05-29 02:53:58.088164539 +0000 UTC m=+88.843479581" lastFinishedPulling="2026-05-29 02:53:59.117526103 +0000 UTC m=+89.872841148" observedRunningTime="2026-05-29 02:53:59.222089686 +0000 UTC m=+89.977404749" watchObservedRunningTime="2026-05-29 02:53:59.223034876 +0000 UTC m=+89.978349940"
May 29 02:53:59.264723 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.264683    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-54688b74b-pvv7w"]
May 29 02:53:59.270332 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.270309    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-54688b74b-pvv7w"]
May 29 02:53:59.430927 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.430849    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-vgtrr"]
May 29 02:53:59.434185 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.434161    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7779bbdd78-vgtrr"
May 29 02:53:59.436955 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.436934    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
May 29 02:53:59.437400 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.437383    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 29 02:53:59.438012 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.437698    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 29 02:53:59.438012 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.437699    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 29 02:53:59.438012 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.437880    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
May 29 02:53:59.438012 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.438003    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-5qx5x\""
May 29 02:53:59.444290 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.444253    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-vgtrr"]
May 29 02:53:59.486227 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.486204    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d948ccb0-1cad-47d5-85ea-69f2c3189425-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-vgtrr\" (UID: \"d948ccb0-1cad-47d5-85ea-69f2c3189425\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-vgtrr"
May 29 02:53:59.486340 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.486237    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d948ccb0-1cad-47d5-85ea-69f2c3189425-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-vgtrr\" (UID: \"d948ccb0-1cad-47d5-85ea-69f2c3189425\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-vgtrr"
May 29 02:53:59.486340 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.486255    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhqdq\" (UniqueName: \"kubernetes.io/projected/d948ccb0-1cad-47d5-85ea-69f2c3189425-kube-api-access-rhqdq\") pod \"prometheus-operator-7779bbdd78-vgtrr\" (UID: \"d948ccb0-1cad-47d5-85ea-69f2c3189425\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-vgtrr"
May 29 02:53:59.486447 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.486357    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d948ccb0-1cad-47d5-85ea-69f2c3189425-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-vgtrr\" (UID: \"d948ccb0-1cad-47d5-85ea-69f2c3189425\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-vgtrr"
May 29 02:53:59.587171 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.587139    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d948ccb0-1cad-47d5-85ea-69f2c3189425-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-vgtrr\" (UID: \"d948ccb0-1cad-47d5-85ea-69f2c3189425\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-vgtrr"
May 29 02:53:59.587310 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.587195    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d948ccb0-1cad-47d5-85ea-69f2c3189425-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-vgtrr\" (UID: \"d948ccb0-1cad-47d5-85ea-69f2c3189425\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-vgtrr"
May 29 02:53:59.587310 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.587227    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rhqdq\" (UniqueName: \"kubernetes.io/projected/d948ccb0-1cad-47d5-85ea-69f2c3189425-kube-api-access-rhqdq\") pod \"prometheus-operator-7779bbdd78-vgtrr\" (UID: \"d948ccb0-1cad-47d5-85ea-69f2c3189425\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-vgtrr"
May 29 02:53:59.587310 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.587304    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d948ccb0-1cad-47d5-85ea-69f2c3189425-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-vgtrr\" (UID: \"d948ccb0-1cad-47d5-85ea-69f2c3189425\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-vgtrr"
May 29 02:53:59.587486 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:59.587405    2559 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
May 29 02:53:59.587486 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:53:59.587461    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d948ccb0-1cad-47d5-85ea-69f2c3189425-prometheus-operator-tls podName:d948ccb0-1cad-47d5-85ea-69f2c3189425 nodeName:}" failed. No retries permitted until 2026-05-29 02:54:00.087443488 +0000 UTC m=+90.842758533 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/d948ccb0-1cad-47d5-85ea-69f2c3189425-prometheus-operator-tls") pod "prometheus-operator-7779bbdd78-vgtrr" (UID: "d948ccb0-1cad-47d5-85ea-69f2c3189425") : secret "prometheus-operator-tls" not found
May 29 02:53:59.587937 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.587913    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d948ccb0-1cad-47d5-85ea-69f2c3189425-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-vgtrr\" (UID: \"d948ccb0-1cad-47d5-85ea-69f2c3189425\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-vgtrr"
May 29 02:53:59.589916 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.589895    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d948ccb0-1cad-47d5-85ea-69f2c3189425-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-vgtrr\" (UID: \"d948ccb0-1cad-47d5-85ea-69f2c3189425\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-vgtrr"
May 29 02:53:59.601065 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.601041    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhqdq\" (UniqueName: \"kubernetes.io/projected/d948ccb0-1cad-47d5-85ea-69f2c3189425-kube-api-access-rhqdq\") pod \"prometheus-operator-7779bbdd78-vgtrr\" (UID: \"d948ccb0-1cad-47d5-85ea-69f2c3189425\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-vgtrr"
May 29 02:53:59.850884 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:53:59.850090    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4942eb9f-01a3-4702-a9cc-be97c76ab116" path="/var/lib/kubelet/pods/4942eb9f-01a3-4702-a9cc-be97c76ab116/volumes"
May 29 02:54:00.091566 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:00.091534    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d948ccb0-1cad-47d5-85ea-69f2c3189425-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-vgtrr\" (UID: \"d948ccb0-1cad-47d5-85ea-69f2c3189425\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-vgtrr"
May 29 02:54:00.093700 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:00.093674    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d948ccb0-1cad-47d5-85ea-69f2c3189425-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-vgtrr\" (UID: \"d948ccb0-1cad-47d5-85ea-69f2c3189425\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-vgtrr"
May 29 02:54:00.211783 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:00.211753    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9thz9" event={"ID":"503df8ce-8b42-4e8e-a59a-92b9a68b979d","Type":"ContainerStarted","Data":"22a93b9a43e0b6ae147b17c7531b2d308defdd9aa5fa187b604dfd4a292a0862"}
May 29 02:54:00.229751 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:00.229705    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9thz9" podStartSLOduration=1.246994994 podStartE2EDuration="3.229689944s" podCreationTimestamp="2026-05-29 02:53:57 +0000 UTC" firstStartedPulling="2026-05-29 02:53:58.156560006 +0000 UTC m=+88.911875054" lastFinishedPulling="2026-05-29 02:54:00.139254963 +0000 UTC m=+90.894570004" observedRunningTime="2026-05-29 02:54:00.229124191 +0000 UTC m=+90.984439256" watchObservedRunningTime="2026-05-29 02:54:00.229689944 +0000 UTC m=+90.985005010"
May 29 02:54:00.345835 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:00.345801    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7779bbdd78-vgtrr"
May 29 02:54:00.460256 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:00.460227    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-vgtrr"]
May 29 02:54:00.463568 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:54:00.463548    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd948ccb0_1cad_47d5_85ea_69f2c3189425.slice/crio-4857421a8d27ae35aefb7731b9844868e7a956a35c18749ad643adbaf776085f WatchSource:0}: Error finding container 4857421a8d27ae35aefb7731b9844868e7a956a35c18749ad643adbaf776085f: Status 404 returned error can't find the container with id 4857421a8d27ae35aefb7731b9844868e7a956a35c18749ad643adbaf776085f
May 29 02:54:01.215960 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:01.215920    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-vgtrr" event={"ID":"d948ccb0-1cad-47d5-85ea-69f2c3189425","Type":"ContainerStarted","Data":"4857421a8d27ae35aefb7731b9844868e7a956a35c18749ad643adbaf776085f"}
May 29 02:54:02.220589 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:02.220552    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-vgtrr" event={"ID":"d948ccb0-1cad-47d5-85ea-69f2c3189425","Type":"ContainerStarted","Data":"ef9f15253f44856c52a1e8b516a446e9fc8bdbb9ea6c7fb65261500a318eba18"}
May 29 02:54:02.220589 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:02.220594    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-vgtrr" event={"ID":"d948ccb0-1cad-47d5-85ea-69f2c3189425","Type":"ContainerStarted","Data":"46c934e13e2b07795f59c2f772ed2147438fdaa5336861d503f5b8f586de0512"}
May 29 02:54:02.239332 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:02.239267    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7779bbdd78-vgtrr" podStartSLOduration=2.040959449 podStartE2EDuration="3.239250416s" podCreationTimestamp="2026-05-29 02:53:59 +0000 UTC" firstStartedPulling="2026-05-29 02:54:00.465803007 +0000 UTC m=+91.221118050" lastFinishedPulling="2026-05-29 02:54:01.664093975 +0000 UTC m=+92.419409017" observedRunningTime="2026-05-29 02:54:02.2388879 +0000 UTC m=+92.994202977" watchObservedRunningTime="2026-05-29 02:54:02.239250416 +0000 UTC m=+92.994565484"
May 29 02:54:03.785642 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.785607    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-b2kdq"]
May 29 02:54:03.789369 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.789348    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-9c959559f-b2kdq"
May 29 02:54:03.791537 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.791513    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
May 29 02:54:03.791745 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.791731    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
May 29 02:54:03.791941 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.791927    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-6cl4h\""
May 29 02:54:03.809605 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.809578    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-b2kdq"]
May 29 02:54:03.811283 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.811261    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5jltk"]
May 29 02:54:03.815215 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.815196    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:03.817864 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.817846    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 29 02:54:03.818018 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.817996    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-m9bld\""
May 29 02:54:03.818142 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.817885    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 29 02:54:03.818399 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.818383    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/97a3a5c7-f634-4f14-9961-1970e366b867-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-b2kdq\" (UID: \"97a3a5c7-f634-4f14-9961-1970e366b867\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-b2kdq"
May 29 02:54:03.818481 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.818450    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97a3a5c7-f634-4f14-9961-1970e366b867-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-b2kdq\" (UID: \"97a3a5c7-f634-4f14-9961-1970e366b867\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-b2kdq"
May 29 02:54:03.818544 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.818501    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4lf9\" (UniqueName: \"kubernetes.io/projected/97a3a5c7-f634-4f14-9961-1970e366b867-kube-api-access-c4lf9\") pod \"openshift-state-metrics-9c959559f-b2kdq\" (UID: \"97a3a5c7-f634-4f14-9961-1970e366b867\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-b2kdq"
May 29 02:54:03.818595 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.818555    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97a3a5c7-f634-4f14-9961-1970e366b867-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-b2kdq\" (UID: \"97a3a5c7-f634-4f14-9961-1970e366b867\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-b2kdq"
May 29 02:54:03.819127 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.819111    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 29 02:54:03.919641 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.919605    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-node-exporter-accelerators-collector-config\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:03.919808 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.919650    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-799hx\" (UniqueName: \"kubernetes.io/projected/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-kube-api-access-799hx\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:03.919808 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.919695    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97a3a5c7-f634-4f14-9961-1970e366b867-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-b2kdq\" (UID: \"97a3a5c7-f634-4f14-9961-1970e366b867\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-b2kdq"
May 29 02:54:03.919808 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.919721    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-root\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:03.919808 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.919755    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-node-exporter-textfile\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:03.919808 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.919781    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/97a3a5c7-f634-4f14-9961-1970e366b867-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-b2kdq\" (UID: \"97a3a5c7-f634-4f14-9961-1970e366b867\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-b2kdq"
May 29 02:54:03.920086 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.919848    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-metrics-client-ca\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:03.920086 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:54:03.919898    2559 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
May 29 02:54:03.920086 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.919938    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-node-exporter-wtmp\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:03.920086 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:54:03.919977    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97a3a5c7-f634-4f14-9961-1970e366b867-openshift-state-metrics-tls podName:97a3a5c7-f634-4f14-9961-1970e366b867 nodeName:}" failed. No retries permitted until 2026-05-29 02:54:04.419944165 +0000 UTC m=+95.175259215 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/97a3a5c7-f634-4f14-9961-1970e366b867-openshift-state-metrics-tls") pod "openshift-state-metrics-9c959559f-b2kdq" (UID: "97a3a5c7-f634-4f14-9961-1970e366b867") : secret "openshift-state-metrics-tls" not found
May 29 02:54:03.920086 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.920038    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-node-exporter-tls\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:03.920086 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.920075    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97a3a5c7-f634-4f14-9961-1970e366b867-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-b2kdq\" (UID: \"97a3a5c7-f634-4f14-9961-1970e366b867\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-b2kdq"
May 29 02:54:03.920375 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.920124    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-sys\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:03.920375 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.920185    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:03.920375 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.920221    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c4lf9\" (UniqueName: \"kubernetes.io/projected/97a3a5c7-f634-4f14-9961-1970e366b867-kube-api-access-c4lf9\") pod \"openshift-state-metrics-9c959559f-b2kdq\" (UID: \"97a3a5c7-f634-4f14-9961-1970e366b867\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-b2kdq"
May 29 02:54:03.920504 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.920424    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97a3a5c7-f634-4f14-9961-1970e366b867-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-b2kdq\" (UID: \"97a3a5c7-f634-4f14-9961-1970e366b867\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-b2kdq"
May 29 02:54:03.922485 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.922465    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97a3a5c7-f634-4f14-9961-1970e366b867-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-b2kdq\" (UID: \"97a3a5c7-f634-4f14-9961-1970e366b867\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-b2kdq"
May 29 02:54:03.931828 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:03.931805    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4lf9\" (UniqueName: \"kubernetes.io/projected/97a3a5c7-f634-4f14-9961-1970e366b867-kube-api-access-c4lf9\") pod \"openshift-state-metrics-9c959559f-b2kdq\" (UID: \"97a3a5c7-f634-4f14-9961-1970e366b867\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-b2kdq"
May 29 02:54:04.020987 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.020943    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-node-exporter-textfile\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:04.021150 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.021019    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-metrics-client-ca\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:04.021150 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.021055    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-node-exporter-wtmp\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:04.021271 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.021181    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-node-exporter-wtmp\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:04.021271 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.021182    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-node-exporter-tls\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:04.021271 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.021226    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-sys\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:04.021423 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:54:04.021275    2559 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
May 29 02:54:04.021423 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.021279    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:04.021423 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.021291    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-sys\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:04.021423 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:54:04.021334    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-node-exporter-tls podName:f09e25bb-880d-47c5-aef4-aa43f3ef48b9 nodeName:}" failed. No retries permitted until 2026-05-29 02:54:04.521313889 +0000 UTC m=+95.276628955 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-node-exporter-tls") pod "node-exporter-5jltk" (UID: "f09e25bb-880d-47c5-aef4-aa43f3ef48b9") : secret "node-exporter-tls" not found
May 29 02:54:04.021423 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.021365    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-node-exporter-accelerators-collector-config\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:04.021423 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.021402    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-799hx\" (UniqueName: \"kubernetes.io/projected/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-kube-api-access-799hx\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:04.021688 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.021433    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-root\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:04.021688 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.021504    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-root\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:04.021688 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.021587    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-metrics-client-ca\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:04.021882 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.021863    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-node-exporter-textfile\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:04.022128 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.022110    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-node-exporter-accelerators-collector-config\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:04.023613 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.023594    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:04.030980 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.030945    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-799hx\" (UniqueName: \"kubernetes.io/projected/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-kube-api-access-799hx\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:04.424534 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.424505    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/97a3a5c7-f634-4f14-9961-1970e366b867-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-b2kdq\" (UID: \"97a3a5c7-f634-4f14-9961-1970e366b867\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-b2kdq"
May 29 02:54:04.426669 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.426651    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/97a3a5c7-f634-4f14-9961-1970e366b867-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-b2kdq\" (UID: \"97a3a5c7-f634-4f14-9961-1970e366b867\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-b2kdq"
May 29 02:54:04.525394 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.525372    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-node-exporter-tls\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:04.527439 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.527422    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f09e25bb-880d-47c5-aef4-aa43f3ef48b9-node-exporter-tls\") pod \"node-exporter-5jltk\" (UID: \"f09e25bb-880d-47c5-aef4-aa43f3ef48b9\") " pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:04.709337 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.709236    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-9c959559f-b2kdq"
May 29 02:54:04.726119 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.726091    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5jltk"
May 29 02:54:04.734890 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:54:04.734858    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf09e25bb_880d_47c5_aef4_aa43f3ef48b9.slice/crio-53255e2c8054d4f40940c80137c5670e82dfdf9f5c7f674b9053a5b9227a06b5 WatchSource:0}: Error finding container 53255e2c8054d4f40940c80137c5670e82dfdf9f5c7f674b9053a5b9227a06b5: Status 404 returned error can't find the container with id 53255e2c8054d4f40940c80137c5670e82dfdf9f5c7f674b9053a5b9227a06b5
May 29 02:54:04.844397 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.844367    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-b2kdq"]
May 29 02:54:04.848302 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:54:04.848271    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97a3a5c7_f634_4f14_9961_1970e366b867.slice/crio-e144f357281b45520c8a53f42f1e93546f447a4f9c2da2bef91da3e4e7e31a52 WatchSource:0}: Error finding container e144f357281b45520c8a53f42f1e93546f447a4f9c2da2bef91da3e4e7e31a52: Status 404 returned error can't find the container with id e144f357281b45520c8a53f42f1e93546f447a4f9c2da2bef91da3e4e7e31a52
May 29 02:54:04.879528 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.879502    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 29 02:54:04.884677 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.884646    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:04.887032 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.886961    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
May 29 02:54:04.887562 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.887325    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
May 29 02:54:04.887562 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.887337    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
May 29 02:54:04.887562 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.887356    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
May 29 02:54:04.887562 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.887387    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
May 29 02:54:04.887562 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.887344    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
May 29 02:54:04.887562 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.887432    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
May 29 02:54:04.887562 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.887348    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
May 29 02:54:04.887562 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.887344    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-6vjhj\""
May 29 02:54:04.887562 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.887391    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
May 29 02:54:04.894461 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.894437    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 29 02:54:04.928854 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.928819    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz4v5\" (UniqueName: \"kubernetes.io/projected/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-kube-api-access-xz4v5\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:04.928937 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.928876    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:04.928937 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.928914    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:04.929060 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.928940    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:04.929060 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.928986    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:04.929060 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.929015    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-web-config\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:04.929060 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.929042    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:04.929248 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.929109    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:04.929248 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.929171    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-config-volume\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:04.929248 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.929241    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:04.929368 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.929276    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:04.929368 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.929330    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:04.929443 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:04.929367    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-config-out\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.030470 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.030436    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.030631 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.030607    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.030673 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.030665    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-config-out\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.030730 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.030716    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xz4v5\" (UniqueName: \"kubernetes.io/projected/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-kube-api-access-xz4v5\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.030803 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.030787    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.030846 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.030833    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.030893 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.030877    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.030942 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.030910    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.031250 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.031186    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.031412 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.031332    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-web-config\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.031412 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.031373    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.031548 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.031420    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.031548 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.031450    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-config-volume\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.031548 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.031492    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.031693 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:54:05.031629    2559 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
May 29 02:54:05.031693 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:54:05.031683    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-main-tls podName:28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2 nodeName:}" failed. No retries permitted until 2026-05-29 02:54:05.53166626 +0000 UTC m=+96.286981302 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2") : secret "alertmanager-main-tls" not found
May 29 02:54:05.031880 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.031417    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.032222 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.032200    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.033497 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.033471    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.033690 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.033670    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-config-out\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.034224 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.034199    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.034572 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.034538    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.034648 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.034629    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.035076 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.035056    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-web-config\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.035279 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.035261    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-config-volume\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.035570 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.035553    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.038789 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.038767    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz4v5\" (UniqueName: \"kubernetes.io/projected/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-kube-api-access-xz4v5\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.229111 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.229020    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5jltk" event={"ID":"f09e25bb-880d-47c5-aef4-aa43f3ef48b9","Type":"ContainerStarted","Data":"53255e2c8054d4f40940c80137c5670e82dfdf9f5c7f674b9053a5b9227a06b5"}
May 29 02:54:05.230657 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.230632    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-b2kdq" event={"ID":"97a3a5c7-f634-4f14-9961-1970e366b867","Type":"ContainerStarted","Data":"126b31653e82179c30d8b49c68d96bbf11db17ed615fcddf05c548c6869e4377"}
May 29 02:54:05.230769 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.230664    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-b2kdq" event={"ID":"97a3a5c7-f634-4f14-9961-1970e366b867","Type":"ContainerStarted","Data":"3925912e3608830fc26dd280e8f1d7f1ac5b193a2d1d08f24bea91549a6f9d05"}
May 29 02:54:05.230769 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.230675    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-b2kdq" event={"ID":"97a3a5c7-f634-4f14-9961-1970e366b867","Type":"ContainerStarted","Data":"e144f357281b45520c8a53f42f1e93546f447a4f9c2da2bef91da3e4e7e31a52"}
May 29 02:54:05.536069 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.535978    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.538573 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.538547    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.798040 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.797956    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 29 02:54:05.951791 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:05.951760    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 29 02:54:05.955104 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:54:05.955076    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28c5ab9d_c989_4741_bca4_9bb9bb7fc1b2.slice/crio-dcae2406005c0eda484ab45f10ccbbf0368e79774cb6903c759f6f7b4447c077 WatchSource:0}: Error finding container dcae2406005c0eda484ab45f10ccbbf0368e79774cb6903c759f6f7b4447c077: Status 404 returned error can't find the container with id dcae2406005c0eda484ab45f10ccbbf0368e79774cb6903c759f6f7b4447c077
May 29 02:54:06.234271 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:06.234240    2559 generic.go:358] "Generic (PLEG): container finished" podID="f09e25bb-880d-47c5-aef4-aa43f3ef48b9" containerID="0bca31dd9de9edc9d8c86886782d02f5c2c47356eac4ff155ebba1a2326643bb" exitCode=0
May 29 02:54:06.234433 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:06.234331    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5jltk" event={"ID":"f09e25bb-880d-47c5-aef4-aa43f3ef48b9","Type":"ContainerDied","Data":"0bca31dd9de9edc9d8c86886782d02f5c2c47356eac4ff155ebba1a2326643bb"}
May 29 02:54:06.235284 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:06.235266    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2","Type":"ContainerStarted","Data":"dcae2406005c0eda484ab45f10ccbbf0368e79774cb6903c759f6f7b4447c077"}
May 29 02:54:07.152511 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:07.152475    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7cca70d-3eb6-4647-9508-894b981687eb-metrics-tls\") pod \"dns-default-9c6sj\" (UID: \"e7cca70d-3eb6-4647-9508-894b981687eb\") " pod="openshift-dns/dns-default-9c6sj"
May 29 02:54:07.154689 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:07.154668    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7cca70d-3eb6-4647-9508-894b981687eb-metrics-tls\") pod \"dns-default-9c6sj\" (UID: \"e7cca70d-3eb6-4647-9508-894b981687eb\") " pod="openshift-dns/dns-default-9c6sj"
May 29 02:54:07.238943 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:07.238881    2559 generic.go:358] "Generic (PLEG): container finished" podID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerID="fc98e254fd9470d4de343421099adfdeedca77c0756270615395e1c3d1c28d15" exitCode=0
May 29 02:54:07.239072 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:07.238996    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2","Type":"ContainerDied","Data":"fc98e254fd9470d4de343421099adfdeedca77c0756270615395e1c3d1c28d15"}
May 29 02:54:07.241021 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:07.240996    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5jltk" event={"ID":"f09e25bb-880d-47c5-aef4-aa43f3ef48b9","Type":"ContainerStarted","Data":"fd68c7985ec99c66aa6356c557a3540d7b4054c8a7c64cf3f39a54b28bc2a67c"}
May 29 02:54:07.241126 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:07.241028    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5jltk" event={"ID":"f09e25bb-880d-47c5-aef4-aa43f3ef48b9","Type":"ContainerStarted","Data":"db75d037065788e758e30e9b1764c7d5475aa2c14cf2116acfbdaabdc6a0507d"}
May 29 02:54:07.242934 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:07.242868    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-b2kdq" event={"ID":"97a3a5c7-f634-4f14-9961-1970e366b867","Type":"ContainerStarted","Data":"b7ea3a69d6936e738a86615166e9affc76a6019bf00cd879c8f2127185acb5c4"}
May 29 02:54:07.243417 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:07.243399    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4sfm2\""
May 29 02:54:07.251241 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:07.251223    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9c6sj"
May 29 02:54:07.252877 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:07.252854    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5ebef60-aa2e-4a66-ad59-307085eae08c-cert\") pod \"ingress-canary-bzt86\" (UID: \"e5ebef60-aa2e-4a66-ad59-307085eae08c\") " pod="openshift-ingress-canary/ingress-canary-bzt86"
May 29 02:54:07.255143 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:07.255115    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5ebef60-aa2e-4a66-ad59-307085eae08c-cert\") pod \"ingress-canary-bzt86\" (UID: \"e5ebef60-aa2e-4a66-ad59-307085eae08c\") " pod="openshift-ingress-canary/ingress-canary-bzt86"
May 29 02:54:07.258896 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:07.258877    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2t8gv\""
May 29 02:54:07.267031 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:07.266992    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bzt86"
May 29 02:54:07.281812 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:07.281733    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5jltk" podStartSLOduration=3.396605507 podStartE2EDuration="4.281716671s" podCreationTimestamp="2026-05-29 02:54:03 +0000 UTC" firstStartedPulling="2026-05-29 02:54:04.737140402 +0000 UTC m=+95.492455443" lastFinishedPulling="2026-05-29 02:54:05.622251352 +0000 UTC m=+96.377566607" observedRunningTime="2026-05-29 02:54:07.281701487 +0000 UTC m=+98.037016575" watchObservedRunningTime="2026-05-29 02:54:07.281716671 +0000 UTC m=+98.037031737"
May 29 02:54:07.385496 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:07.385448    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-9c959559f-b2kdq" podStartSLOduration=3.119672626 podStartE2EDuration="4.385428119s" podCreationTimestamp="2026-05-29 02:54:03 +0000 UTC" firstStartedPulling="2026-05-29 02:54:04.993099098 +0000 UTC m=+95.748414143" lastFinishedPulling="2026-05-29 02:54:06.25885458 +0000 UTC m=+97.014169636" observedRunningTime="2026-05-29 02:54:07.30355628 +0000 UTC m=+98.058871347" watchObservedRunningTime="2026-05-29 02:54:07.385428119 +0000 UTC m=+98.140743182"
May 29 02:54:07.386460 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:07.386437    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9c6sj"]
May 29 02:54:07.389171 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:54:07.389144    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7cca70d_3eb6_4647_9508_894b981687eb.slice/crio-4fc3322d86a4d4e0831e1dfbb9ba13ab8a044207beb2613e1c1e620611caf8e3 WatchSource:0}: Error finding container 4fc3322d86a4d4e0831e1dfbb9ba13ab8a044207beb2613e1c1e620611caf8e3: Status 404 returned error can't find the container with id 4fc3322d86a4d4e0831e1dfbb9ba13ab8a044207beb2613e1c1e620611caf8e3
May 29 02:54:07.401753 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:07.401732    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bzt86"]
May 29 02:54:07.404219 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:54:07.404196    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5ebef60_aa2e_4a66_ad59_307085eae08c.slice/crio-44b2232cd5fabe271a83b517c7a96f55301e84795cc534d32b3f0bb0d62961c6 WatchSource:0}: Error finding container 44b2232cd5fabe271a83b517c7a96f55301e84795cc534d32b3f0bb0d62961c6: Status 404 returned error can't find the container with id 44b2232cd5fabe271a83b517c7a96f55301e84795cc534d32b3f0bb0d62961c6
May 29 02:54:08.247931 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:08.247860    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9c6sj" event={"ID":"e7cca70d-3eb6-4647-9508-894b981687eb","Type":"ContainerStarted","Data":"4fc3322d86a4d4e0831e1dfbb9ba13ab8a044207beb2613e1c1e620611caf8e3"}
May 29 02:54:08.250467 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:08.250430    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bzt86" event={"ID":"e5ebef60-aa2e-4a66-ad59-307085eae08c","Type":"ContainerStarted","Data":"44b2232cd5fabe271a83b517c7a96f55301e84795cc534d32b3f0bb0d62961c6"}
May 29 02:54:10.083143 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.083107    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 02:54:10.089254 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.089228    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.091338 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.091317    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
May 29 02:54:10.091448 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.091341    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
May 29 02:54:10.091448 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.091317    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
May 29 02:54:10.091568 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.091467    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
May 29 02:54:10.094822 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.093888    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-f6mrds684j17m\""
May 29 02:54:10.094822 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.094211    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
May 29 02:54:10.094822 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.094641    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
May 29 02:54:10.095077 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.094884    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
May 29 02:54:10.095077 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.094915    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
May 29 02:54:10.095077 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.094942    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
May 29 02:54:10.095222 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.095206    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
May 29 02:54:10.095286 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.095268    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
May 29 02:54:10.095586 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.095568    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-qxtlb\""
May 29 02:54:10.096892 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.096874    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
May 29 02:54:10.099811 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.099788    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
May 29 02:54:10.103554 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.103529    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 02:54:10.156679 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.156586    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fnzpl"
May 29 02:54:10.181259 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.181232    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.181259 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.181262    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e91e8e40-8028-423b-9fe2-8260065aa3c7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.181470 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.181287    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-config\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.181470 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.181354    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.181470 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.181414    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.181470 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.181441    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.181637 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.181472    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.181637 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.181504    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.181637 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.181565    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.181637 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.181608    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.181803 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.181637    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.181803 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.181665    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.181803 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.181691    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ghnd\" (UniqueName: \"kubernetes.io/projected/e91e8e40-8028-423b-9fe2-8260065aa3c7-kube-api-access-9ghnd\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.181803 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.181728    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-web-config\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.181803 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.181757    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e91e8e40-8028-423b-9fe2-8260065aa3c7-config-out\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.182044 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.181810    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.182044 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.181875    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e91e8e40-8028-423b-9fe2-8260065aa3c7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.182044 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.181948    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.260665 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.260615    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2","Type":"ContainerStarted","Data":"c195e7b119a8ecb3e9a9e35688f9feac7cb1a139abb678c5a6fcbaf1c790a0a0"}
May 29 02:54:10.260825 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.260681    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2","Type":"ContainerStarted","Data":"8f151290dd43230972a330ea4b2a4172df91d7ef5293ee07a20cf81ca924f082"}
May 29 02:54:10.260825 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.260697    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2","Type":"ContainerStarted","Data":"1e0c43a65ddd98cbe7e54be691f2d17ba04625be539c261cac1ee0976e847c3e"}
May 29 02:54:10.260825 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.260709    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2","Type":"ContainerStarted","Data":"bf2a64652b66e21597eccfcbbb1f3e8345edbea65a3e4dcecbe24e19673a7d59"}
May 29 02:54:10.260825 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.260721    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2","Type":"ContainerStarted","Data":"9a2493189e9f351f91e209ccb45e71648c9bd4b3cc7f34630466a9a1f94b0edc"}
May 29 02:54:10.262248 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.262216    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9c6sj" event={"ID":"e7cca70d-3eb6-4647-9508-894b981687eb","Type":"ContainerStarted","Data":"59ec8d40e9a0311b8e28b5bc5eb16e954f07be082b1375e30f4c3bc1e75c410c"}
May 29 02:54:10.262378 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.262251    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9c6sj" event={"ID":"e7cca70d-3eb6-4647-9508-894b981687eb","Type":"ContainerStarted","Data":"7a47e3b25c4f408c1ce97fc0729c20b7956e24f9170bb65c2c97068a7af6187d"}
May 29 02:54:10.262378 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.262352    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9c6sj"
May 29 02:54:10.263432 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.263400    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bzt86" event={"ID":"e5ebef60-aa2e-4a66-ad59-307085eae08c","Type":"ContainerStarted","Data":"5beb1c1b4bd77268c3ea365561f3f494cd28a298f765e3ada17c208a94a2efff"}
May 29 02:54:10.280401 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.280344    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-9c6sj" podStartSLOduration=65.036836978 podStartE2EDuration="1m7.280328952s" podCreationTimestamp="2026-05-29 02:53:03 +0000 UTC" firstStartedPulling="2026-05-29 02:54:07.390929964 +0000 UTC m=+98.146245007" lastFinishedPulling="2026-05-29 02:54:09.634421939 +0000 UTC m=+100.389736981" observedRunningTime="2026-05-29 02:54:10.280309449 +0000 UTC m=+101.035624513" watchObservedRunningTime="2026-05-29 02:54:10.280328952 +0000 UTC m=+101.035644017"
May 29 02:54:10.282539 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.282520    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.282623 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.282546    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e91e8e40-8028-423b-9fe2-8260065aa3c7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.282623 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.282573    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-config\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.282787 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.282770    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.282841 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.282825    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.282881 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.282853    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.282917 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.282893    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.282991 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.282921    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.282991 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.282954    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.283091 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.283012    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.283091 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.283018    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e91e8e40-8028-423b-9fe2-8260065aa3c7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.283091 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.283042    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.283234 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.283088    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.283234 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.283122    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9ghnd\" (UniqueName: \"kubernetes.io/projected/e91e8e40-8028-423b-9fe2-8260065aa3c7-kube-api-access-9ghnd\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.283234 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.283159    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-web-config\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.283234 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.283192    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e91e8e40-8028-423b-9fe2-8260065aa3c7-config-out\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.283234 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.283231    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.283483 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.283262    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e91e8e40-8028-423b-9fe2-8260065aa3c7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.283483 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.283293    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.283587 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.283572    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.283640 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.283604    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.284024 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.283952    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.284506 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.284487    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.286736 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.285990    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.286736 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.286379    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-config\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.286736 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.286700    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e91e8e40-8028-423b-9fe2-8260065aa3c7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.287320 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.287286    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.287885 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.287819    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.288503 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.288474    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-web-config\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.289150 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.289134    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.290485 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.290428    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.290645 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.290623    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.290876 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.290848    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e91e8e40-8028-423b-9fe2-8260065aa3c7-config-out\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.291177 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.291157    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.291546 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.291527    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.294480 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.294463    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ghnd\" (UniqueName: \"kubernetes.io/projected/e91e8e40-8028-423b-9fe2-8260065aa3c7-kube-api-access-9ghnd\") pod \"prometheus-k8s-0\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.299342 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.299297    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bzt86" podStartSLOduration=65.065296982 podStartE2EDuration="1m7.299283726s" podCreationTimestamp="2026-05-29 02:53:03 +0000 UTC" firstStartedPulling="2026-05-29 02:54:07.405788496 +0000 UTC m=+98.161103538" lastFinishedPulling="2026-05-29 02:54:09.639775238 +0000 UTC m=+100.395090282" observedRunningTime="2026-05-29 02:54:10.297366991 +0000 UTC m=+101.052682066" watchObservedRunningTime="2026-05-29 02:54:10.299283726 +0000 UTC m=+101.054598790"
May 29 02:54:10.403327 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.403295    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:10.559280 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:10.559174    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 02:54:10.561833 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:54:10.561796    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode91e8e40_8028_423b_9fe2_8260065aa3c7.slice/crio-261dd06a266f9013799c9293cec684156ff353ea3e277e77ecfc43589819fbf5 WatchSource:0}: Error finding container 261dd06a266f9013799c9293cec684156ff353ea3e277e77ecfc43589819fbf5: Status 404 returned error can't find the container with id 261dd06a266f9013799c9293cec684156ff353ea3e277e77ecfc43589819fbf5
May 29 02:54:11.267957 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:11.267926    2559 generic.go:358] "Generic (PLEG): container finished" podID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerID="b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e" exitCode=0
May 29 02:54:11.268356 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:11.267999    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e91e8e40-8028-423b-9fe2-8260065aa3c7","Type":"ContainerDied","Data":"b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e"}
May 29 02:54:11.268356 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:11.268033    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e91e8e40-8028-423b-9fe2-8260065aa3c7","Type":"ContainerStarted","Data":"261dd06a266f9013799c9293cec684156ff353ea3e277e77ecfc43589819fbf5"}
May 29 02:54:11.272017 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:11.271990    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2","Type":"ContainerStarted","Data":"c2390bacc6a71161030985c925463a1cb7745063c7342de5548b4ff9daf1e420"}
May 29 02:54:11.321863 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:11.321819    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.4144965369999998 podStartE2EDuration="7.321806841s" podCreationTimestamp="2026-05-29 02:54:04 +0000 UTC" firstStartedPulling="2026-05-29 02:54:05.957574485 +0000 UTC m=+96.712889528" lastFinishedPulling="2026-05-29 02:54:10.864884786 +0000 UTC m=+101.620199832" observedRunningTime="2026-05-29 02:54:11.320322694 +0000 UTC m=+102.075637757" watchObservedRunningTime="2026-05-29 02:54:11.321806841 +0000 UTC m=+102.077121909"
May 29 02:54:15.288912 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:15.288875    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e91e8e40-8028-423b-9fe2-8260065aa3c7","Type":"ContainerStarted","Data":"5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca"}
May 29 02:54:15.288912 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:15.288915    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e91e8e40-8028-423b-9fe2-8260065aa3c7","Type":"ContainerStarted","Data":"7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50"}
May 29 02:54:17.296544 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:17.296512    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e91e8e40-8028-423b-9fe2-8260065aa3c7","Type":"ContainerStarted","Data":"628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1"}
May 29 02:54:17.296544 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:17.296544    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e91e8e40-8028-423b-9fe2-8260065aa3c7","Type":"ContainerStarted","Data":"284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850"}
May 29 02:54:17.296544 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:17.296554    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e91e8e40-8028-423b-9fe2-8260065aa3c7","Type":"ContainerStarted","Data":"828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167"}
May 29 02:54:17.296993 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:17.296562    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e91e8e40-8028-423b-9fe2-8260065aa3c7","Type":"ContainerStarted","Data":"87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2"}
May 29 02:54:17.336142 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:17.336093    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.356784535 podStartE2EDuration="7.336080709s" podCreationTimestamp="2026-05-29 02:54:10 +0000 UTC" firstStartedPulling="2026-05-29 02:54:11.269200393 +0000 UTC m=+102.024515437" lastFinishedPulling="2026-05-29 02:54:16.248496567 +0000 UTC m=+107.003811611" observedRunningTime="2026-05-29 02:54:17.335065955 +0000 UTC m=+108.090381254" watchObservedRunningTime="2026-05-29 02:54:17.336080709 +0000 UTC m=+108.091395772"
May 29 02:54:20.273945 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:20.273919    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9c6sj"
May 29 02:54:20.403560 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:20.403533    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:54:38.614823 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:38.614785    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs\") pod \"network-metrics-daemon-r4cd8\" (UID: \"757d1101-4ef1-451f-b984-012545775d40\") " pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:54:38.617140 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:38.617118    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/757d1101-4ef1-451f-b984-012545775d40-metrics-certs\") pod \"network-metrics-daemon-r4cd8\" (UID: \"757d1101-4ef1-451f-b984-012545775d40\") " pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:54:38.766885 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:38.766856    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n4b9g\""
May 29 02:54:38.771563 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:38.771547    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4cd8"
May 29 02:54:38.901856 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:38.901785    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r4cd8"]
May 29 02:54:38.904628 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:54:38.904598    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod757d1101_4ef1_451f_b984_012545775d40.slice/crio-1f05488a06a1056a26549fb0b4a08d69eb433b2cd2f70df4f2567bc42941f93c WatchSource:0}: Error finding container 1f05488a06a1056a26549fb0b4a08d69eb433b2cd2f70df4f2567bc42941f93c: Status 404 returned error can't find the container with id 1f05488a06a1056a26549fb0b4a08d69eb433b2cd2f70df4f2567bc42941f93c
May 29 02:54:39.358918 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:39.358886    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r4cd8" event={"ID":"757d1101-4ef1-451f-b984-012545775d40","Type":"ContainerStarted","Data":"1f05488a06a1056a26549fb0b4a08d69eb433b2cd2f70df4f2567bc42941f93c"}
May 29 02:54:40.362682 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:40.362651    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r4cd8" event={"ID":"757d1101-4ef1-451f-b984-012545775d40","Type":"ContainerStarted","Data":"fb8a050e1e07cb973364d97595d3703df5ea124048da884c87980e36a72abfbc"}
May 29 02:54:40.362682 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:40.362686    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r4cd8" event={"ID":"757d1101-4ef1-451f-b984-012545775d40","Type":"ContainerStarted","Data":"2f4c8084812118aac300e699c86deee5f479e6ce54eb29b53e24d43f9611bd9b"}
May 29 02:54:40.384662 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:40.384620    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-r4cd8" podStartSLOduration=130.511252345 podStartE2EDuration="2m11.384606174s" podCreationTimestamp="2026-05-29 02:52:29 +0000 UTC" firstStartedPulling="2026-05-29 02:54:38.906701468 +0000 UTC m=+129.662016510" lastFinishedPulling="2026-05-29 02:54:39.780055288 +0000 UTC m=+130.535370339" observedRunningTime="2026-05-29 02:54:40.382829108 +0000 UTC m=+131.138144173" watchObservedRunningTime="2026-05-29 02:54:40.384606174 +0000 UTC m=+131.139921237"
May 29 02:54:48.385337 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:48.385256    2559 generic.go:358] "Generic (PLEG): container finished" podID="d897b3db-7602-4711-96e7-b21b92e4b006" containerID="09f05cd6235f03d9f39a5aa80f5593c3819963dde3a9f9f94153dd826518cf5b" exitCode=0
May 29 02:54:48.385337 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:48.385323    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-z2zvs" event={"ID":"d897b3db-7602-4711-96e7-b21b92e4b006","Type":"ContainerDied","Data":"09f05cd6235f03d9f39a5aa80f5593c3819963dde3a9f9f94153dd826518cf5b"}
May 29 02:54:48.385704 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:48.385614    2559 scope.go:117] "RemoveContainer" containerID="09f05cd6235f03d9f39a5aa80f5593c3819963dde3a9f9f94153dd826518cf5b"
May 29 02:54:49.389822 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:49.389787    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-z2zvs" event={"ID":"d897b3db-7602-4711-96e7-b21b92e4b006","Type":"ContainerStarted","Data":"c13221737e903508f04d4a0b51ab06ec87d47a80c21f0fdae0d680c548d2eb87"}
May 29 02:54:51.525064 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:54:51.525030    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bzt86_e5ebef60-aa2e-4a66-ad59-307085eae08c/serve-healthcheck-canary/0.log"
May 29 02:55:10.403927 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:10.403889    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:10.422449 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:10.422425    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:10.468271 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:10.468249    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:24.121262 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:24.121133    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 29 02:55:24.121806 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:24.121754    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerName="alertmanager" containerID="cri-o://9a2493189e9f351f91e209ccb45e71648c9bd4b3cc7f34630466a9a1f94b0edc" gracePeriod=120
May 29 02:55:24.121877 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:24.121811    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerName="config-reloader" containerID="cri-o://bf2a64652b66e21597eccfcbbb1f3e8345edbea65a3e4dcecbe24e19673a7d59" gracePeriod=120
May 29 02:55:24.121938 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:24.121780    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerName="kube-rbac-proxy-metric" containerID="cri-o://c195e7b119a8ecb3e9a9e35688f9feac7cb1a139abb678c5a6fcbaf1c790a0a0" gracePeriod=120
May 29 02:55:24.121938 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:24.121847    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerName="kube-rbac-proxy-web" containerID="cri-o://1e0c43a65ddd98cbe7e54be691f2d17ba04625be539c261cac1ee0976e847c3e" gracePeriod=120
May 29 02:55:24.122068 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:24.121997    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerName="kube-rbac-proxy" containerID="cri-o://8f151290dd43230972a330ea4b2a4172df91d7ef5293ee07a20cf81ca924f082" gracePeriod=120
May 29 02:55:24.122113 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:24.122060    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerName="prom-label-proxy" containerID="cri-o://c2390bacc6a71161030985c925463a1cb7745063c7342de5548b4ff9daf1e420" gracePeriod=120
May 29 02:55:24.495877 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:24.495796    2559 generic.go:358] "Generic (PLEG): container finished" podID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerID="c2390bacc6a71161030985c925463a1cb7745063c7342de5548b4ff9daf1e420" exitCode=0
May 29 02:55:24.495877 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:24.495824    2559 generic.go:358] "Generic (PLEG): container finished" podID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerID="c195e7b119a8ecb3e9a9e35688f9feac7cb1a139abb678c5a6fcbaf1c790a0a0" exitCode=0
May 29 02:55:24.495877 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:24.495830    2559 generic.go:358] "Generic (PLEG): container finished" podID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerID="8f151290dd43230972a330ea4b2a4172df91d7ef5293ee07a20cf81ca924f082" exitCode=0
May 29 02:55:24.495877 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:24.495836    2559 generic.go:358] "Generic (PLEG): container finished" podID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerID="bf2a64652b66e21597eccfcbbb1f3e8345edbea65a3e4dcecbe24e19673a7d59" exitCode=0
May 29 02:55:24.495877 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:24.495841    2559 generic.go:358] "Generic (PLEG): container finished" podID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerID="9a2493189e9f351f91e209ccb45e71648c9bd4b3cc7f34630466a9a1f94b0edc" exitCode=0
May 29 02:55:24.496162 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:24.495867    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2","Type":"ContainerDied","Data":"c2390bacc6a71161030985c925463a1cb7745063c7342de5548b4ff9daf1e420"}
May 29 02:55:24.496162 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:24.495908    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2","Type":"ContainerDied","Data":"c195e7b119a8ecb3e9a9e35688f9feac7cb1a139abb678c5a6fcbaf1c790a0a0"}
May 29 02:55:24.496162 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:24.495920    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2","Type":"ContainerDied","Data":"8f151290dd43230972a330ea4b2a4172df91d7ef5293ee07a20cf81ca924f082"}
May 29 02:55:24.496162 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:24.495931    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2","Type":"ContainerDied","Data":"bf2a64652b66e21597eccfcbbb1f3e8345edbea65a3e4dcecbe24e19673a7d59"}
May 29 02:55:24.496162 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:24.495944    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2","Type":"ContainerDied","Data":"9a2493189e9f351f91e209ccb45e71648c9bd4b3cc7f34630466a9a1f94b0edc"}
May 29 02:55:25.360889 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.360868    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.473775 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.473706    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-kube-rbac-proxy\") pod \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") "
May 29 02:55:25.473775 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.473735    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-main-tls\") pod \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") "
May 29 02:55:25.473775 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.473754    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-config-volume\") pod \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") "
May 29 02:55:25.473775 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.473772    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-kube-rbac-proxy-web\") pod \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") "
May 29 02:55:25.474100 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.473797    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-tls-assets\") pod \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") "
May 29 02:55:25.474100 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.473896    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-alertmanager-main-db\") pod \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") "
May 29 02:55:25.474100 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.473944    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-metrics-client-ca\") pod \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") "
May 29 02:55:25.474100 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.474022    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-cluster-tls-config\") pod \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") "
May 29 02:55:25.474100 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.474054    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-config-out\") pod \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") "
May 29 02:55:25.474335 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.474104    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-web-config\") pod \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") "
May 29 02:55:25.474335 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.474149    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xz4v5\" (UniqueName: \"kubernetes.io/projected/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-kube-api-access-xz4v5\") pod \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") "
May 29 02:55:25.474335 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.474191    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-alertmanager-trusted-ca-bundle\") pod \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") "
May 29 02:55:25.474335 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.474224    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\" (UID: \"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2\") "
May 29 02:55:25.474335 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.474289    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" (UID: "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 02:55:25.474335 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.474312    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" (UID: "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 02:55:25.474532 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.474499    2559 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-alertmanager-main-db\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:25.474532 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.474519    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-metrics-client-ca\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:25.475731 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.475697    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" (UID: "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 02:55:25.476647 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.476620    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" (UID: "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 02:55:25.477080 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.477045    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" (UID: "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:55:25.477339 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.477315    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-config-volume" (OuterVolumeSpecName: "config-volume") pod "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" (UID: "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:55:25.477818 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.477788    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" (UID: "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:55:25.477907 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.477811    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" (UID: "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:55:25.478078 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.478047    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" (UID: "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:55:25.478301 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.478278    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-kube-api-access-xz4v5" (OuterVolumeSpecName: "kube-api-access-xz4v5") pod "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" (UID: "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2"). InnerVolumeSpecName "kube-api-access-xz4v5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 02:55:25.478484 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.478465    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-config-out" (OuterVolumeSpecName: "config-out") pod "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" (UID: "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 02:55:25.481606 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.481582    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" (UID: "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:55:25.487691 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.487669    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-web-config" (OuterVolumeSpecName: "web-config") pod "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" (UID: "28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:55:25.501284 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.501262    2559 generic.go:358] "Generic (PLEG): container finished" podID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerID="1e0c43a65ddd98cbe7e54be691f2d17ba04625be539c261cac1ee0976e847c3e" exitCode=0
May 29 02:55:25.501379 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.501341    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2","Type":"ContainerDied","Data":"1e0c43a65ddd98cbe7e54be691f2d17ba04625be539c261cac1ee0976e847c3e"}
May 29 02:55:25.501379 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.501376    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2","Type":"ContainerDied","Data":"dcae2406005c0eda484ab45f10ccbbf0368e79774cb6903c759f6f7b4447c077"}
May 29 02:55:25.501488 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.501392    2559 scope.go:117] "RemoveContainer" containerID="c2390bacc6a71161030985c925463a1cb7745063c7342de5548b4ff9daf1e420"
May 29 02:55:25.501488 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.501396    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.508498 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.508469    2559 scope.go:117] "RemoveContainer" containerID="c195e7b119a8ecb3e9a9e35688f9feac7cb1a139abb678c5a6fcbaf1c790a0a0"
May 29 02:55:25.515413 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.515392    2559 scope.go:117] "RemoveContainer" containerID="8f151290dd43230972a330ea4b2a4172df91d7ef5293ee07a20cf81ca924f082"
May 29 02:55:25.521505 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.521488    2559 scope.go:117] "RemoveContainer" containerID="1e0c43a65ddd98cbe7e54be691f2d17ba04625be539c261cac1ee0976e847c3e"
May 29 02:55:25.524550 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.524529    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 29 02:55:25.527770 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.527754    2559 scope.go:117] "RemoveContainer" containerID="bf2a64652b66e21597eccfcbbb1f3e8345edbea65a3e4dcecbe24e19673a7d59"
May 29 02:55:25.530380 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.530358    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 29 02:55:25.534543 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.534529    2559 scope.go:117] "RemoveContainer" containerID="9a2493189e9f351f91e209ccb45e71648c9bd4b3cc7f34630466a9a1f94b0edc"
May 29 02:55:25.540459 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.540443    2559 scope.go:117] "RemoveContainer" containerID="fc98e254fd9470d4de343421099adfdeedca77c0756270615395e1c3d1c28d15"
May 29 02:55:25.546215 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.546198    2559 scope.go:117] "RemoveContainer" containerID="c2390bacc6a71161030985c925463a1cb7745063c7342de5548b4ff9daf1e420"
May 29 02:55:25.546502 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:55:25.546470    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2390bacc6a71161030985c925463a1cb7745063c7342de5548b4ff9daf1e420\": container with ID starting with c2390bacc6a71161030985c925463a1cb7745063c7342de5548b4ff9daf1e420 not found: ID does not exist" containerID="c2390bacc6a71161030985c925463a1cb7745063c7342de5548b4ff9daf1e420"
May 29 02:55:25.546605 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.546504    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2390bacc6a71161030985c925463a1cb7745063c7342de5548b4ff9daf1e420"} err="failed to get container status \"c2390bacc6a71161030985c925463a1cb7745063c7342de5548b4ff9daf1e420\": rpc error: code = NotFound desc = could not find container \"c2390bacc6a71161030985c925463a1cb7745063c7342de5548b4ff9daf1e420\": container with ID starting with c2390bacc6a71161030985c925463a1cb7745063c7342de5548b4ff9daf1e420 not found: ID does not exist"
May 29 02:55:25.546605 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.546538    2559 scope.go:117] "RemoveContainer" containerID="c195e7b119a8ecb3e9a9e35688f9feac7cb1a139abb678c5a6fcbaf1c790a0a0"
May 29 02:55:25.546837 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:55:25.546812    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c195e7b119a8ecb3e9a9e35688f9feac7cb1a139abb678c5a6fcbaf1c790a0a0\": container with ID starting with c195e7b119a8ecb3e9a9e35688f9feac7cb1a139abb678c5a6fcbaf1c790a0a0 not found: ID does not exist" containerID="c195e7b119a8ecb3e9a9e35688f9feac7cb1a139abb678c5a6fcbaf1c790a0a0"
May 29 02:55:25.546945 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.546845    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c195e7b119a8ecb3e9a9e35688f9feac7cb1a139abb678c5a6fcbaf1c790a0a0"} err="failed to get container status \"c195e7b119a8ecb3e9a9e35688f9feac7cb1a139abb678c5a6fcbaf1c790a0a0\": rpc error: code = NotFound desc = could not find container \"c195e7b119a8ecb3e9a9e35688f9feac7cb1a139abb678c5a6fcbaf1c790a0a0\": container with ID starting with c195e7b119a8ecb3e9a9e35688f9feac7cb1a139abb678c5a6fcbaf1c790a0a0 not found: ID does not exist"
May 29 02:55:25.546945 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.546866    2559 scope.go:117] "RemoveContainer" containerID="8f151290dd43230972a330ea4b2a4172df91d7ef5293ee07a20cf81ca924f082"
May 29 02:55:25.547205 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:55:25.547138    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f151290dd43230972a330ea4b2a4172df91d7ef5293ee07a20cf81ca924f082\": container with ID starting with 8f151290dd43230972a330ea4b2a4172df91d7ef5293ee07a20cf81ca924f082 not found: ID does not exist" containerID="8f151290dd43230972a330ea4b2a4172df91d7ef5293ee07a20cf81ca924f082"
May 29 02:55:25.547205 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.547170    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f151290dd43230972a330ea4b2a4172df91d7ef5293ee07a20cf81ca924f082"} err="failed to get container status \"8f151290dd43230972a330ea4b2a4172df91d7ef5293ee07a20cf81ca924f082\": rpc error: code = NotFound desc = could not find container \"8f151290dd43230972a330ea4b2a4172df91d7ef5293ee07a20cf81ca924f082\": container with ID starting with 8f151290dd43230972a330ea4b2a4172df91d7ef5293ee07a20cf81ca924f082 not found: ID does not exist"
May 29 02:55:25.547205 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.547191    2559 scope.go:117] "RemoveContainer" containerID="1e0c43a65ddd98cbe7e54be691f2d17ba04625be539c261cac1ee0976e847c3e"
May 29 02:55:25.547440 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:55:25.547423    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e0c43a65ddd98cbe7e54be691f2d17ba04625be539c261cac1ee0976e847c3e\": container with ID starting with 1e0c43a65ddd98cbe7e54be691f2d17ba04625be539c261cac1ee0976e847c3e not found: ID does not exist" containerID="1e0c43a65ddd98cbe7e54be691f2d17ba04625be539c261cac1ee0976e847c3e"
May 29 02:55:25.547481 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.547453    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e0c43a65ddd98cbe7e54be691f2d17ba04625be539c261cac1ee0976e847c3e"} err="failed to get container status \"1e0c43a65ddd98cbe7e54be691f2d17ba04625be539c261cac1ee0976e847c3e\": rpc error: code = NotFound desc = could not find container \"1e0c43a65ddd98cbe7e54be691f2d17ba04625be539c261cac1ee0976e847c3e\": container with ID starting with 1e0c43a65ddd98cbe7e54be691f2d17ba04625be539c261cac1ee0976e847c3e not found: ID does not exist"
May 29 02:55:25.547481 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.547468    2559 scope.go:117] "RemoveContainer" containerID="bf2a64652b66e21597eccfcbbb1f3e8345edbea65a3e4dcecbe24e19673a7d59"
May 29 02:55:25.547658 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:55:25.547644    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf2a64652b66e21597eccfcbbb1f3e8345edbea65a3e4dcecbe24e19673a7d59\": container with ID starting with bf2a64652b66e21597eccfcbbb1f3e8345edbea65a3e4dcecbe24e19673a7d59 not found: ID does not exist" containerID="bf2a64652b66e21597eccfcbbb1f3e8345edbea65a3e4dcecbe24e19673a7d59"
May 29 02:55:25.547722 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.547662    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf2a64652b66e21597eccfcbbb1f3e8345edbea65a3e4dcecbe24e19673a7d59"} err="failed to get container status \"bf2a64652b66e21597eccfcbbb1f3e8345edbea65a3e4dcecbe24e19673a7d59\": rpc error: code = NotFound desc = could not find container \"bf2a64652b66e21597eccfcbbb1f3e8345edbea65a3e4dcecbe24e19673a7d59\": container with ID starting with bf2a64652b66e21597eccfcbbb1f3e8345edbea65a3e4dcecbe24e19673a7d59 not found: ID does not exist"
May 29 02:55:25.547722 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.547673    2559 scope.go:117] "RemoveContainer" containerID="9a2493189e9f351f91e209ccb45e71648c9bd4b3cc7f34630466a9a1f94b0edc"
May 29 02:55:25.547841 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:55:25.547827    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a2493189e9f351f91e209ccb45e71648c9bd4b3cc7f34630466a9a1f94b0edc\": container with ID starting with 9a2493189e9f351f91e209ccb45e71648c9bd4b3cc7f34630466a9a1f94b0edc not found: ID does not exist" containerID="9a2493189e9f351f91e209ccb45e71648c9bd4b3cc7f34630466a9a1f94b0edc"
May 29 02:55:25.547884 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.547843    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a2493189e9f351f91e209ccb45e71648c9bd4b3cc7f34630466a9a1f94b0edc"} err="failed to get container status \"9a2493189e9f351f91e209ccb45e71648c9bd4b3cc7f34630466a9a1f94b0edc\": rpc error: code = NotFound desc = could not find container \"9a2493189e9f351f91e209ccb45e71648c9bd4b3cc7f34630466a9a1f94b0edc\": container with ID starting with 9a2493189e9f351f91e209ccb45e71648c9bd4b3cc7f34630466a9a1f94b0edc not found: ID does not exist"
May 29 02:55:25.547884 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.547855    2559 scope.go:117] "RemoveContainer" containerID="fc98e254fd9470d4de343421099adfdeedca77c0756270615395e1c3d1c28d15"
May 29 02:55:25.548218 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:55:25.548199    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc98e254fd9470d4de343421099adfdeedca77c0756270615395e1c3d1c28d15\": container with ID starting with fc98e254fd9470d4de343421099adfdeedca77c0756270615395e1c3d1c28d15 not found: ID does not exist" containerID="fc98e254fd9470d4de343421099adfdeedca77c0756270615395e1c3d1c28d15"
May 29 02:55:25.548286 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.548226    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc98e254fd9470d4de343421099adfdeedca77c0756270615395e1c3d1c28d15"} err="failed to get container status \"fc98e254fd9470d4de343421099adfdeedca77c0756270615395e1c3d1c28d15\": rpc error: code = NotFound desc = could not find container \"fc98e254fd9470d4de343421099adfdeedca77c0756270615395e1c3d1c28d15\": container with ID starting with fc98e254fd9470d4de343421099adfdeedca77c0756270615395e1c3d1c28d15 not found: ID does not exist"
May 29 02:55:25.555566 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.555546    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 29 02:55:25.555853 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.555840    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerName="kube-rbac-proxy"
May 29 02:55:25.555912 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.555855    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerName="kube-rbac-proxy"
May 29 02:55:25.555912 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.555871    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerName="config-reloader"
May 29 02:55:25.555912 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.555877    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerName="config-reloader"
May 29 02:55:25.555912 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.555883    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerName="alertmanager"
May 29 02:55:25.555912 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.555889    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerName="alertmanager"
May 29 02:55:25.555912 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.555895    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerName="prom-label-proxy"
May 29 02:55:25.555912 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.555901    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerName="prom-label-proxy"
May 29 02:55:25.556150 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.555924    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerName="kube-rbac-proxy-metric"
May 29 02:55:25.556150 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.555931    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerName="kube-rbac-proxy-metric"
May 29 02:55:25.556150 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.555945    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerName="init-config-reloader"
May 29 02:55:25.556150 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.555953    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerName="init-config-reloader"
May 29 02:55:25.556150 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.555958    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerName="kube-rbac-proxy-web"
May 29 02:55:25.556150 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.555977    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerName="kube-rbac-proxy-web"
May 29 02:55:25.556150 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.556031    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerName="kube-rbac-proxy-metric"
May 29 02:55:25.556150 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.556038    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerName="kube-rbac-proxy-web"
May 29 02:55:25.556150 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.556044    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerName="prom-label-proxy"
May 29 02:55:25.556150 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.556053    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerName="kube-rbac-proxy"
May 29 02:55:25.556150 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.556060    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerName="alertmanager"
May 29 02:55:25.556150 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.556066    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" containerName="config-reloader"
May 29 02:55:25.560865 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.560848    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.563239 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.563077    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
May 29 02:55:25.563239 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.563103    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
May 29 02:55:25.563239 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.563125    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
May 29 02:55:25.563239 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.563129    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-6vjhj\""
May 29 02:55:25.563239 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.563141    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
May 29 02:55:25.563239 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.563182    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
May 29 02:55:25.563239 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.563079    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
May 29 02:55:25.563239 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.563129    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
May 29 02:55:25.563239 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.563134    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
May 29 02:55:25.568390 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.568373    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
May 29 02:55:25.575348 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.575314    2559 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-tls-assets\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:25.575348 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.575340    2559 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-cluster-tls-config\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:25.575506 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.575354    2559 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-config-out\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:25.575506 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.575370    2559 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-web-config\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:25.575506 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.575383    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xz4v5\" (UniqueName: \"kubernetes.io/projected/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-kube-api-access-xz4v5\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:25.575506 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.575398    2559 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:25.575506 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.575414    2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:25.575506 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.575431    2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:25.575506 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.575446    2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-main-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:25.575506 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.575459    2559 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-config-volume\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:25.575506 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.575474    2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:25.575832 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.575817    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 29 02:55:25.676057 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.676035    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f8649618-fbf6-4f7b-9281-a1942009267b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.676162 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.676062    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rnkz\" (UniqueName: \"kubernetes.io/projected/f8649618-fbf6-4f7b-9281-a1942009267b-kube-api-access-8rnkz\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.676162 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.676084    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f8649618-fbf6-4f7b-9281-a1942009267b-config-volume\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.676162 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.676136    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f8649618-fbf6-4f7b-9281-a1942009267b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.676162 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.676157    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f8649618-fbf6-4f7b-9281-a1942009267b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.676291 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.676179    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f8649618-fbf6-4f7b-9281-a1942009267b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.676291 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.676247    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f8649618-fbf6-4f7b-9281-a1942009267b-web-config\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.676291 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.676285    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f8649618-fbf6-4f7b-9281-a1942009267b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.676377 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.676305    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8649618-fbf6-4f7b-9281-a1942009267b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.676377 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.676327    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8649618-fbf6-4f7b-9281-a1942009267b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.676377 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.676356    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f8649618-fbf6-4f7b-9281-a1942009267b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.676463 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.676389    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f8649618-fbf6-4f7b-9281-a1942009267b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.676463 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.676410    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f8649618-fbf6-4f7b-9281-a1942009267b-config-out\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.776913 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.776852    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f8649618-fbf6-4f7b-9281-a1942009267b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.776913 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.776881    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8rnkz\" (UniqueName: \"kubernetes.io/projected/f8649618-fbf6-4f7b-9281-a1942009267b-kube-api-access-8rnkz\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.776913 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.776899    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f8649618-fbf6-4f7b-9281-a1942009267b-config-volume\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.777100 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.776920    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f8649618-fbf6-4f7b-9281-a1942009267b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.777136 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.777121    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f8649618-fbf6-4f7b-9281-a1942009267b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.777181 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.777164    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f8649618-fbf6-4f7b-9281-a1942009267b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.777233 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.777211    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f8649618-fbf6-4f7b-9281-a1942009267b-web-config\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.777282 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.777239    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f8649618-fbf6-4f7b-9281-a1942009267b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.777282 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.777265    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8649618-fbf6-4f7b-9281-a1942009267b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.777378 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.777299    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8649618-fbf6-4f7b-9281-a1942009267b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.777378 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.777352    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f8649618-fbf6-4f7b-9281-a1942009267b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.777483 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.777381    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f8649618-fbf6-4f7b-9281-a1942009267b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.777483 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.777411    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f8649618-fbf6-4f7b-9281-a1942009267b-config-out\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.777647 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.777609    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f8649618-fbf6-4f7b-9281-a1942009267b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.778218 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.778195    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8649618-fbf6-4f7b-9281-a1942009267b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.779809 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.779776    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f8649618-fbf6-4f7b-9281-a1942009267b-config-out\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.779941 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.779919    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f8649618-fbf6-4f7b-9281-a1942009267b-config-volume\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.780031 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.779980    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f8649618-fbf6-4f7b-9281-a1942009267b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.780031 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.779998    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f8649618-fbf6-4f7b-9281-a1942009267b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.780156 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.780135    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f8649618-fbf6-4f7b-9281-a1942009267b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.780518 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.780490    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f8649618-fbf6-4f7b-9281-a1942009267b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.780688 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.780671    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f8649618-fbf6-4f7b-9281-a1942009267b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.780755 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.780692    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f8649618-fbf6-4f7b-9281-a1942009267b-web-config\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.780789 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.780760    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8649618-fbf6-4f7b-9281-a1942009267b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.781643 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.781625    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f8649618-fbf6-4f7b-9281-a1942009267b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.784463 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.784447    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rnkz\" (UniqueName: \"kubernetes.io/projected/f8649618-fbf6-4f7b-9281-a1942009267b-kube-api-access-8rnkz\") pod \"alertmanager-main-0\" (UID: \"f8649618-fbf6-4f7b-9281-a1942009267b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:25.848213 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.848180    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2" path="/var/lib/kubelet/pods/28c5ab9d-c989-4741-bca4-9bb9bb7fc1b2/volumes"
May 29 02:55:25.870289 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:25.870266    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 29 02:55:26.000219 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:26.000191    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 29 02:55:26.002732 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:55:26.002710    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8649618_fbf6_4f7b_9281_a1942009267b.slice/crio-e83559f09e6d427a142db1b08a7a345efa107f30fc94d66af3d247987c229cfe WatchSource:0}: Error finding container e83559f09e6d427a142db1b08a7a345efa107f30fc94d66af3d247987c229cfe: Status 404 returned error can't find the container with id e83559f09e6d427a142db1b08a7a345efa107f30fc94d66af3d247987c229cfe
May 29 02:55:26.508906 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:26.508259    2559 generic.go:358] "Generic (PLEG): container finished" podID="f8649618-fbf6-4f7b-9281-a1942009267b" containerID="18252a6be985e5773ad1e314c160a8e7dac9221cb16d6d2f244915d8e70e7b1b" exitCode=0
May 29 02:55:26.508906 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:26.508327    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f8649618-fbf6-4f7b-9281-a1942009267b","Type":"ContainerDied","Data":"18252a6be985e5773ad1e314c160a8e7dac9221cb16d6d2f244915d8e70e7b1b"}
May 29 02:55:26.508906 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:26.508358    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f8649618-fbf6-4f7b-9281-a1942009267b","Type":"ContainerStarted","Data":"e83559f09e6d427a142db1b08a7a345efa107f30fc94d66af3d247987c229cfe"}
May 29 02:55:27.518801 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:27.518768    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f8649618-fbf6-4f7b-9281-a1942009267b","Type":"ContainerStarted","Data":"579f2bd1723d8d9b7f02dc1bea0cac09a1c7c5a21f1bf8f0febc38a1f573387d"}
May 29 02:55:27.518801 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:27.518799    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f8649618-fbf6-4f7b-9281-a1942009267b","Type":"ContainerStarted","Data":"2731d9fa3bda1f8946df357db3f78a49b46b6a88aa35aff12992074d2e08793a"}
May 29 02:55:27.518801 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:27.518809    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f8649618-fbf6-4f7b-9281-a1942009267b","Type":"ContainerStarted","Data":"6dabba672afdfffe14f44c578cc2ed90f000eb4696b6b2cd6f6e8c3880fd3bc6"}
May 29 02:55:27.519393 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:27.518817    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f8649618-fbf6-4f7b-9281-a1942009267b","Type":"ContainerStarted","Data":"6a1d28b77489352b056a31420410df4caaac57578d93e2bf0fd48fe0b830b5b6"}
May 29 02:55:27.519393 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:27.518825    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f8649618-fbf6-4f7b-9281-a1942009267b","Type":"ContainerStarted","Data":"cfc897773e70cf1c90d9e58a9214983dbac5155ad6ce328a8ca8809d61d89b3a"}
May 29 02:55:27.519393 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:27.518833    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f8649618-fbf6-4f7b-9281-a1942009267b","Type":"ContainerStarted","Data":"b48e2cff17479e81a738332240b8f86a982a9cdbeacf3e0ec46f8d942a31a824"}
May 29 02:55:27.544330 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:27.544282    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.544268027 podStartE2EDuration="2.544268027s" podCreationTimestamp="2026-05-29 02:55:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 02:55:27.542234202 +0000 UTC m=+178.297549265" watchObservedRunningTime="2026-05-29 02:55:27.544268027 +0000 UTC m=+178.299583090"
May 29 02:55:28.512951 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.512919    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 02:55:28.513400 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.513366    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerName="prometheus" containerID="cri-o://7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50" gracePeriod=600
May 29 02:55:28.513543 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.513392    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerName="kube-rbac-proxy" containerID="cri-o://284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850" gracePeriod=600
May 29 02:55:28.513543 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.513409    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerName="thanos-sidecar" containerID="cri-o://87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2" gracePeriod=600
May 29 02:55:28.513543 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.513451    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerName="config-reloader" containerID="cri-o://5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca" gracePeriod=600
May 29 02:55:28.513543 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.513517    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerName="kube-rbac-proxy-web" containerID="cri-o://828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167" gracePeriod=600
May 29 02:55:28.513543 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.513530    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerName="kube-rbac-proxy-thanos" containerID="cri-o://628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1" gracePeriod=600
May 29 02:55:28.752807 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.752785    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:28.903741 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.903711    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ghnd\" (UniqueName: \"kubernetes.io/projected/e91e8e40-8028-423b-9fe2-8260065aa3c7-kube-api-access-9ghnd\") pod \"e91e8e40-8028-423b-9fe2-8260065aa3c7\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") "
May 29 02:55:28.903929 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.903757    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"e91e8e40-8028-423b-9fe2-8260065aa3c7\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") "
May 29 02:55:28.903929 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.903775    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-kube-rbac-proxy\") pod \"e91e8e40-8028-423b-9fe2-8260065aa3c7\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") "
May 29 02:55:28.903929 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.903790    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e91e8e40-8028-423b-9fe2-8260065aa3c7-tls-assets\") pod \"e91e8e40-8028-423b-9fe2-8260065aa3c7\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") "
May 29 02:55:28.903929 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.903851    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-configmap-kubelet-serving-ca-bundle\") pod \"e91e8e40-8028-423b-9fe2-8260065aa3c7\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") "
May 29 02:55:28.903929 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.903886    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-config\") pod \"e91e8e40-8028-423b-9fe2-8260065aa3c7\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") "
May 29 02:55:28.903929 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.903912    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e91e8e40-8028-423b-9fe2-8260065aa3c7-config-out\") pod \"e91e8e40-8028-423b-9fe2-8260065aa3c7\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") "
May 29 02:55:28.904279 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.903949    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-prometheus-trusted-ca-bundle\") pod \"e91e8e40-8028-423b-9fe2-8260065aa3c7\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") "
May 29 02:55:28.904279 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.904001    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-configmap-serving-certs-ca-bundle\") pod \"e91e8e40-8028-423b-9fe2-8260065aa3c7\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") "
May 29 02:55:28.904279 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.904032    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-web-config\") pod \"e91e8e40-8028-423b-9fe2-8260065aa3c7\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") "
May 29 02:55:28.904279 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.904068    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-prometheus-k8s-tls\") pod \"e91e8e40-8028-423b-9fe2-8260065aa3c7\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") "
May 29 02:55:28.904279 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.904091    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-grpc-tls\") pod \"e91e8e40-8028-423b-9fe2-8260065aa3c7\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") "
May 29 02:55:28.904279 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.904127    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e91e8e40-8028-423b-9fe2-8260065aa3c7-prometheus-k8s-db\") pod \"e91e8e40-8028-423b-9fe2-8260065aa3c7\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") "
May 29 02:55:28.904279 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.904151    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-prometheus-k8s-rulefiles-0\") pod \"e91e8e40-8028-423b-9fe2-8260065aa3c7\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") "
May 29 02:55:28.904279 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.904188    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"e91e8e40-8028-423b-9fe2-8260065aa3c7\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") "
May 29 02:55:28.904279 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.904227    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-configmap-metrics-client-ca\") pod \"e91e8e40-8028-423b-9fe2-8260065aa3c7\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") "
May 29 02:55:28.904279 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.904259    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-thanos-prometheus-http-client-file\") pod \"e91e8e40-8028-423b-9fe2-8260065aa3c7\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") "
May 29 02:55:28.904750 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.904274    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "e91e8e40-8028-423b-9fe2-8260065aa3c7" (UID: "e91e8e40-8028-423b-9fe2-8260065aa3c7"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 02:55:28.904750 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.904324    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-metrics-client-certs\") pod \"e91e8e40-8028-423b-9fe2-8260065aa3c7\" (UID: \"e91e8e40-8028-423b-9fe2-8260065aa3c7\") "
May 29 02:55:28.904750 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.904597    2559 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:28.906116 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.906087    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e91e8e40-8028-423b-9fe2-8260065aa3c7-kube-api-access-9ghnd" (OuterVolumeSpecName: "kube-api-access-9ghnd") pod "e91e8e40-8028-423b-9fe2-8260065aa3c7" (UID: "e91e8e40-8028-423b-9fe2-8260065aa3c7"). InnerVolumeSpecName "kube-api-access-9ghnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 02:55:28.906300 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.906278    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e91e8e40-8028-423b-9fe2-8260065aa3c7-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e91e8e40-8028-423b-9fe2-8260065aa3c7" (UID: "e91e8e40-8028-423b-9fe2-8260065aa3c7"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 02:55:28.907368 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.906388    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "e91e8e40-8028-423b-9fe2-8260065aa3c7" (UID: "e91e8e40-8028-423b-9fe2-8260065aa3c7"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:55:28.907368 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.906405    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "e91e8e40-8028-423b-9fe2-8260065aa3c7" (UID: "e91e8e40-8028-423b-9fe2-8260065aa3c7"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:55:28.907368 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.906466    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "e91e8e40-8028-423b-9fe2-8260065aa3c7" (UID: "e91e8e40-8028-423b-9fe2-8260065aa3c7"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 02:55:28.907368 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.906646    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "e91e8e40-8028-423b-9fe2-8260065aa3c7" (UID: "e91e8e40-8028-423b-9fe2-8260065aa3c7"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 02:55:28.907368 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.906780    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-config" (OuterVolumeSpecName: "config") pod "e91e8e40-8028-423b-9fe2-8260065aa3c7" (UID: "e91e8e40-8028-423b-9fe2-8260065aa3c7"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:55:28.907368 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.906770    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "e91e8e40-8028-423b-9fe2-8260065aa3c7" (UID: "e91e8e40-8028-423b-9fe2-8260065aa3c7"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:55:28.907368 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.907157    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "e91e8e40-8028-423b-9fe2-8260065aa3c7" (UID: "e91e8e40-8028-423b-9fe2-8260065aa3c7"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 02:55:28.907368 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.907307    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "e91e8e40-8028-423b-9fe2-8260065aa3c7" (UID: "e91e8e40-8028-423b-9fe2-8260065aa3c7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 02:55:28.907798 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.907416    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "e91e8e40-8028-423b-9fe2-8260065aa3c7" (UID: "e91e8e40-8028-423b-9fe2-8260065aa3c7"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:55:28.907798 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.907611    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e91e8e40-8028-423b-9fe2-8260065aa3c7-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "e91e8e40-8028-423b-9fe2-8260065aa3c7" (UID: "e91e8e40-8028-423b-9fe2-8260065aa3c7"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 02:55:28.908260 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.908238    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "e91e8e40-8028-423b-9fe2-8260065aa3c7" (UID: "e91e8e40-8028-423b-9fe2-8260065aa3c7"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:55:28.908721 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.908693    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e91e8e40-8028-423b-9fe2-8260065aa3c7-config-out" (OuterVolumeSpecName: "config-out") pod "e91e8e40-8028-423b-9fe2-8260065aa3c7" (UID: "e91e8e40-8028-423b-9fe2-8260065aa3c7"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 02:55:28.908936 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.908921    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "e91e8e40-8028-423b-9fe2-8260065aa3c7" (UID: "e91e8e40-8028-423b-9fe2-8260065aa3c7"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:55:28.909177 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.909151    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "e91e8e40-8028-423b-9fe2-8260065aa3c7" (UID: "e91e8e40-8028-423b-9fe2-8260065aa3c7"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:55:28.918485 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:28.918456    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-web-config" (OuterVolumeSpecName: "web-config") pod "e91e8e40-8028-423b-9fe2-8260065aa3c7" (UID: "e91e8e40-8028-423b-9fe2-8260065aa3c7"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:55:29.005677 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.005651    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ghnd\" (UniqueName: \"kubernetes.io/projected/e91e8e40-8028-423b-9fe2-8260065aa3c7-kube-api-access-9ghnd\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:29.005677 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.005673    2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:29.005819 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.005684    2559 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-kube-rbac-proxy\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:29.005819 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.005694    2559 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e91e8e40-8028-423b-9fe2-8260065aa3c7-tls-assets\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:29.005819 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.005702    2559 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-config\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:29.005819 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.005711    2559 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e91e8e40-8028-423b-9fe2-8260065aa3c7-config-out\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:29.005819 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.005720    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-prometheus-trusted-ca-bundle\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:29.005819 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.005729    2559 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:29.005819 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.005737    2559 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-web-config\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:29.005819 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.005746    2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-prometheus-k8s-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:29.005819 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.005754    2559 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-grpc-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:29.005819 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.005762    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e91e8e40-8028-423b-9fe2-8260065aa3c7-prometheus-k8s-db\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:29.005819 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.005770    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:29.005819 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.005778    2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:29.005819 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.005788    2559 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e91e8e40-8028-423b-9fe2-8260065aa3c7-configmap-metrics-client-ca\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:29.005819 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.005796    2559 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-thanos-prometheus-http-client-file\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:29.005819 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.005805    2559 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e91e8e40-8028-423b-9fe2-8260065aa3c7-secret-metrics-client-certs\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 02:55:29.531997 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.531949    2559 generic.go:358] "Generic (PLEG): container finished" podID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerID="628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1" exitCode=0
May 29 02:55:29.531997 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.531996    2559 generic.go:358] "Generic (PLEG): container finished" podID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerID="284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850" exitCode=0
May 29 02:55:29.532147 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.532006    2559 generic.go:358] "Generic (PLEG): container finished" podID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerID="828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167" exitCode=0
May 29 02:55:29.532147 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.532016    2559 generic.go:358] "Generic (PLEG): container finished" podID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerID="87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2" exitCode=0
May 29 02:55:29.532147 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.532026    2559 generic.go:358] "Generic (PLEG): container finished" podID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerID="5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca" exitCode=0
May 29 02:55:29.532147 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.532038    2559 generic.go:358] "Generic (PLEG): container finished" podID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerID="7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50" exitCode=0
May 29 02:55:29.532147 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.532049    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e91e8e40-8028-423b-9fe2-8260065aa3c7","Type":"ContainerDied","Data":"628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1"}
May 29 02:55:29.532147 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.532090    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e91e8e40-8028-423b-9fe2-8260065aa3c7","Type":"ContainerDied","Data":"284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850"}
May 29 02:55:29.532147 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.532101    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e91e8e40-8028-423b-9fe2-8260065aa3c7","Type":"ContainerDied","Data":"828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167"}
May 29 02:55:29.532147 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.532111    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e91e8e40-8028-423b-9fe2-8260065aa3c7","Type":"ContainerDied","Data":"87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2"}
May 29 02:55:29.532147 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.532127    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e91e8e40-8028-423b-9fe2-8260065aa3c7","Type":"ContainerDied","Data":"5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca"}
May 29 02:55:29.532147 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.532132    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.532147 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.532141    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e91e8e40-8028-423b-9fe2-8260065aa3c7","Type":"ContainerDied","Data":"7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50"}
May 29 02:55:29.532590 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.532157    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e91e8e40-8028-423b-9fe2-8260065aa3c7","Type":"ContainerDied","Data":"261dd06a266f9013799c9293cec684156ff353ea3e277e77ecfc43589819fbf5"}
May 29 02:55:29.532590 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.532180    2559 scope.go:117] "RemoveContainer" containerID="628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1"
May 29 02:55:29.547669 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.547649    2559 scope.go:117] "RemoveContainer" containerID="284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850"
May 29 02:55:29.554329 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.554308    2559 scope.go:117] "RemoveContainer" containerID="828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167"
May 29 02:55:29.560754 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.560730    2559 scope.go:117] "RemoveContainer" containerID="87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2"
May 29 02:55:29.561531 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.561508    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 02:55:29.567246 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.567229    2559 scope.go:117] "RemoveContainer" containerID="5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca"
May 29 02:55:29.570191 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.570168    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 02:55:29.574017 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.573999    2559 scope.go:117] "RemoveContainer" containerID="7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50"
May 29 02:55:29.580396 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.580381    2559 scope.go:117] "RemoveContainer" containerID="b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e"
May 29 02:55:29.586221 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.586204    2559 scope.go:117] "RemoveContainer" containerID="628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1"
May 29 02:55:29.586454 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:55:29.586434    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1\": container with ID starting with 628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1 not found: ID does not exist" containerID="628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1"
May 29 02:55:29.586521 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.586466    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1"} err="failed to get container status \"628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1\": rpc error: code = NotFound desc = could not find container \"628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1\": container with ID starting with 628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1 not found: ID does not exist"
May 29 02:55:29.586521 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.586492    2559 scope.go:117] "RemoveContainer" containerID="284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850"
May 29 02:55:29.586715 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:55:29.586699    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850\": container with ID starting with 284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850 not found: ID does not exist" containerID="284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850"
May 29 02:55:29.586756 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.586721    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850"} err="failed to get container status \"284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850\": rpc error: code = NotFound desc = could not find container \"284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850\": container with ID starting with 284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850 not found: ID does not exist"
May 29 02:55:29.586756 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.586737    2559 scope.go:117] "RemoveContainer" containerID="828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167"
May 29 02:55:29.586937 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:55:29.586920    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167\": container with ID starting with 828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167 not found: ID does not exist" containerID="828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167"
May 29 02:55:29.587007 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.586943    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167"} err="failed to get container status \"828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167\": rpc error: code = NotFound desc = could not find container \"828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167\": container with ID starting with 828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167 not found: ID does not exist"
May 29 02:55:29.587007 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.586978    2559 scope.go:117] "RemoveContainer" containerID="87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2"
May 29 02:55:29.587238 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:55:29.587223    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2\": container with ID starting with 87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2 not found: ID does not exist" containerID="87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2"
May 29 02:55:29.587292 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.587243    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2"} err="failed to get container status \"87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2\": rpc error: code = NotFound desc = could not find container \"87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2\": container with ID starting with 87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2 not found: ID does not exist"
May 29 02:55:29.587292 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.587263    2559 scope.go:117] "RemoveContainer" containerID="5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca"
May 29 02:55:29.587480 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:55:29.587464    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca\": container with ID starting with 5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca not found: ID does not exist" containerID="5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca"
May 29 02:55:29.587534 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.587485    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca"} err="failed to get container status \"5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca\": rpc error: code = NotFound desc = could not find container \"5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca\": container with ID starting with 5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca not found: ID does not exist"
May 29 02:55:29.587534 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.587504    2559 scope.go:117] "RemoveContainer" containerID="7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50"
May 29 02:55:29.587695 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:55:29.587681    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50\": container with ID starting with 7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50 not found: ID does not exist" containerID="7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50"
May 29 02:55:29.587750 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.587701    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50"} err="failed to get container status \"7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50\": rpc error: code = NotFound desc = could not find container \"7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50\": container with ID starting with 7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50 not found: ID does not exist"
May 29 02:55:29.587750 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.587720    2559 scope.go:117] "RemoveContainer" containerID="b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e"
May 29 02:55:29.587930 ip-10-0-137-122 kubenswrapper[2559]: E0529 02:55:29.587914    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e\": container with ID starting with b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e not found: ID does not exist" containerID="b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e"
May 29 02:55:29.588035 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.587936    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e"} err="failed to get container status \"b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e\": rpc error: code = NotFound desc = could not find container \"b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e\": container with ID starting with b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e not found: ID does not exist"
May 29 02:55:29.588035 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.588021    2559 scope.go:117] "RemoveContainer" containerID="628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1"
May 29 02:55:29.588296 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.588279    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1"} err="failed to get container status \"628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1\": rpc error: code = NotFound desc = could not find container \"628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1\": container with ID starting with 628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1 not found: ID does not exist"
May 29 02:55:29.588362 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.588296    2559 scope.go:117] "RemoveContainer" containerID="284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850"
May 29 02:55:29.588530 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.588483    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850"} err="failed to get container status \"284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850\": rpc error: code = NotFound desc = could not find container \"284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850\": container with ID starting with 284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850 not found: ID does not exist"
May 29 02:55:29.588530 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.588504    2559 scope.go:117] "RemoveContainer" containerID="828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167"
May 29 02:55:29.588736 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.588716    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167"} err="failed to get container status \"828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167\": rpc error: code = NotFound desc = could not find container \"828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167\": container with ID starting with 828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167 not found: ID does not exist"
May 29 02:55:29.588804 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.588737    2559 scope.go:117] "RemoveContainer" containerID="87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2"
May 29 02:55:29.588959 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.588940    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2"} err="failed to get container status \"87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2\": rpc error: code = NotFound desc = could not find container \"87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2\": container with ID starting with 87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2 not found: ID does not exist"
May 29 02:55:29.589028 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.588960    2559 scope.go:117] "RemoveContainer" containerID="5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca"
May 29 02:55:29.589195 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.589178    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca"} err="failed to get container status \"5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca\": rpc error: code = NotFound desc = could not find container \"5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca\": container with ID starting with 5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca not found: ID does not exist"
May 29 02:55:29.589239 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.589195    2559 scope.go:117] "RemoveContainer" containerID="7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50"
May 29 02:55:29.589396 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.589379    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50"} err="failed to get container status \"7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50\": rpc error: code = NotFound desc = could not find container \"7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50\": container with ID starting with 7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50 not found: ID does not exist"
May 29 02:55:29.589456 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.589398    2559 scope.go:117] "RemoveContainer" containerID="b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e"
May 29 02:55:29.589635 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.589617    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e"} err="failed to get container status \"b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e\": rpc error: code = NotFound desc = could not find container \"b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e\": container with ID starting with b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e not found: ID does not exist"
May 29 02:55:29.589673 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.589637    2559 scope.go:117] "RemoveContainer" containerID="628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1"
May 29 02:55:29.589836 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.589820    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1"} err="failed to get container status \"628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1\": rpc error: code = NotFound desc = could not find container \"628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1\": container with ID starting with 628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1 not found: ID does not exist"
May 29 02:55:29.589892 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.589836    2559 scope.go:117] "RemoveContainer" containerID="284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850"
May 29 02:55:29.590076 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.590059    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850"} err="failed to get container status \"284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850\": rpc error: code = NotFound desc = could not find container \"284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850\": container with ID starting with 284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850 not found: ID does not exist"
May 29 02:55:29.590135 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.590077    2559 scope.go:117] "RemoveContainer" containerID="828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167"
May 29 02:55:29.590285 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.590269    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167"} err="failed to get container status \"828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167\": rpc error: code = NotFound desc = could not find container \"828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167\": container with ID starting with 828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167 not found: ID does not exist"
May 29 02:55:29.590330 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.590285    2559 scope.go:117] "RemoveContainer" containerID="87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2"
May 29 02:55:29.590494 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.590477    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2"} err="failed to get container status \"87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2\": rpc error: code = NotFound desc = could not find container \"87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2\": container with ID starting with 87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2 not found: ID does not exist"
May 29 02:55:29.590561 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.590496    2559 scope.go:117] "RemoveContainer" containerID="5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca"
May 29 02:55:29.590718 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.590702    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca"} err="failed to get container status \"5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca\": rpc error: code = NotFound desc = could not find container \"5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca\": container with ID starting with 5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca not found: ID does not exist"
May 29 02:55:29.590779 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.590720    2559 scope.go:117] "RemoveContainer" containerID="7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50"
May 29 02:55:29.590906 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.590885    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50"} err="failed to get container status \"7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50\": rpc error: code = NotFound desc = could not find container \"7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50\": container with ID starting with 7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50 not found: ID does not exist"
May 29 02:55:29.590962 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.590908    2559 scope.go:117] "RemoveContainer" containerID="b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e"
May 29 02:55:29.591090 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.591072    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e"} err="failed to get container status \"b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e\": rpc error: code = NotFound desc = could not find container \"b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e\": container with ID starting with b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e not found: ID does not exist"
May 29 02:55:29.591155 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.591092    2559 scope.go:117] "RemoveContainer" containerID="628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1"
May 29 02:55:29.591274 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.591256    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1"} err="failed to get container status \"628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1\": rpc error: code = NotFound desc = could not find container \"628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1\": container with ID starting with 628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1 not found: ID does not exist"
May 29 02:55:29.591312 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.591274    2559 scope.go:117] "RemoveContainer" containerID="284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850"
May 29 02:55:29.591456 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.591438    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850"} err="failed to get container status \"284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850\": rpc error: code = NotFound desc = could not find container \"284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850\": container with ID starting with 284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850 not found: ID does not exist"
May 29 02:55:29.591507 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.591458    2559 scope.go:117] "RemoveContainer" containerID="828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167"
May 29 02:55:29.591678 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.591659    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167"} err="failed to get container status \"828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167\": rpc error: code = NotFound desc = could not find container \"828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167\": container with ID starting with 828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167 not found: ID does not exist"
May 29 02:55:29.591737 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.591679    2559 scope.go:117] "RemoveContainer" containerID="87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2"
May 29 02:55:29.591864 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.591848    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2"} err="failed to get container status \"87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2\": rpc error: code = NotFound desc = could not find container \"87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2\": container with ID starting with 87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2 not found: ID does not exist"
May 29 02:55:29.591922 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.591865    2559 scope.go:117] "RemoveContainer" containerID="5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca"
May 29 02:55:29.592132 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.592115    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca"} err="failed to get container status \"5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca\": rpc error: code = NotFound desc = could not find container \"5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca\": container with ID starting with 5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca not found: ID does not exist"
May 29 02:55:29.592200 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.592133    2559 scope.go:117] "RemoveContainer" containerID="7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50"
May 29 02:55:29.592332 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.592313    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50"} err="failed to get container status \"7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50\": rpc error: code = NotFound desc = could not find container \"7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50\": container with ID starting with 7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50 not found: ID does not exist"
May 29 02:55:29.592389 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.592333    2559 scope.go:117] "RemoveContainer" containerID="b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e"
May 29 02:55:29.592526 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.592510    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e"} err="failed to get container status \"b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e\": rpc error: code = NotFound desc = could not find container \"b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e\": container with ID starting with b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e not found: ID does not exist"
May 29 02:55:29.592584 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.592527    2559 scope.go:117] "RemoveContainer" containerID="628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1"
May 29 02:55:29.592710 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.592685    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1"} err="failed to get container status \"628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1\": rpc error: code = NotFound desc = could not find container \"628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1\": container with ID starting with 628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1 not found: ID does not exist"
May 29 02:55:29.592710 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.592709    2559 scope.go:117] "RemoveContainer" containerID="284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850"
May 29 02:55:29.592950 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.592932    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850"} err="failed to get container status \"284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850\": rpc error: code = NotFound desc = could not find container \"284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850\": container with ID starting with 284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850 not found: ID does not exist"
May 29 02:55:29.593027 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.592950    2559 scope.go:117] "RemoveContainer" containerID="828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167"
May 29 02:55:29.593175 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.593155    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167"} err="failed to get container status \"828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167\": rpc error: code = NotFound desc = could not find container \"828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167\": container with ID starting with 828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167 not found: ID does not exist"
May 29 02:55:29.593212 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.593177    2559 scope.go:117] "RemoveContainer" containerID="87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2"
May 29 02:55:29.593397 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.593378    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2"} err="failed to get container status \"87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2\": rpc error: code = NotFound desc = could not find container \"87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2\": container with ID starting with 87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2 not found: ID does not exist"
May 29 02:55:29.593470 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.593398    2559 scope.go:117] "RemoveContainer" containerID="5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca"
May 29 02:55:29.593589 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.593571    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca"} err="failed to get container status \"5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca\": rpc error: code = NotFound desc = could not find container \"5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca\": container with ID starting with 5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca not found: ID does not exist"
May 29 02:55:29.593644 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.593590    2559 scope.go:117] "RemoveContainer" containerID="7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50"
May 29 02:55:29.593783 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.593766    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50"} err="failed to get container status \"7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50\": rpc error: code = NotFound desc = could not find container \"7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50\": container with ID starting with 7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50 not found: ID does not exist"
May 29 02:55:29.593846 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.593784    2559 scope.go:117] "RemoveContainer" containerID="b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e"
May 29 02:55:29.593998 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.593959    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e"} err="failed to get container status \"b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e\": rpc error: code = NotFound desc = could not find container \"b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e\": container with ID starting with b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e not found: ID does not exist"
May 29 02:55:29.594068 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.593999    2559 scope.go:117] "RemoveContainer" containerID="628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1"
May 29 02:55:29.594189 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.594174    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1"} err="failed to get container status \"628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1\": rpc error: code = NotFound desc = could not find container \"628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1\": container with ID starting with 628780196a444c60b1a17fa38a97dba4460d972c68db4cb088624d3c9f0e13e1 not found: ID does not exist"
May 29 02:55:29.594247 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.594191    2559 scope.go:117] "RemoveContainer" containerID="284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850"
May 29 02:55:29.594370 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.594355    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850"} err="failed to get container status \"284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850\": rpc error: code = NotFound desc = could not find container \"284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850\": container with ID starting with 284f3e56c1837f2ad47548660d305464507df3c181a8549a8d8b88aa6f711850 not found: ID does not exist"
May 29 02:55:29.594433 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.594372    2559 scope.go:117] "RemoveContainer" containerID="828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167"
May 29 02:55:29.594555 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.594538    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167"} err="failed to get container status \"828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167\": rpc error: code = NotFound desc = could not find container \"828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167\": container with ID starting with 828808c3611e2978be942977b4115784286d847d8e76e56217e37389c8f15167 not found: ID does not exist"
May 29 02:55:29.594621 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.594556    2559 scope.go:117] "RemoveContainer" containerID="87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2"
May 29 02:55:29.594771 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.594752    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2"} err="failed to get container status \"87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2\": rpc error: code = NotFound desc = could not find container \"87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2\": container with ID starting with 87da6c516171cc03bf2e38d5c4b643fc9c6a9657f1af3f8ab93e615184d082b2 not found: ID does not exist"
May 29 02:55:29.594835 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.594772    2559 scope.go:117] "RemoveContainer" containerID="5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca"
May 29 02:55:29.594985 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.594942    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca"} err="failed to get container status \"5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca\": rpc error: code = NotFound desc = could not find container \"5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca\": container with ID starting with 5caddc67381207c6eb1cd6649141a5b051a5c4374a77d25df70c5c4bcd1d7bca not found: ID does not exist"
May 29 02:55:29.594985 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.594977    2559 scope.go:117] "RemoveContainer" containerID="7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50"
May 29 02:55:29.595204 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.595187    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50"} err="failed to get container status \"7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50\": rpc error: code = NotFound desc = could not find container \"7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50\": container with ID starting with 7fbdad518ef0dd9c0310a3f9b47098651dc4e4013227be0924c5606826e6aa50 not found: ID does not exist"
May 29 02:55:29.595246 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.595205    2559 scope.go:117] "RemoveContainer" containerID="b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e"
May 29 02:55:29.595434 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.595417    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e"} err="failed to get container status \"b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e\": rpc error: code = NotFound desc = could not find container \"b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e\": container with ID starting with b51e2dbcdee356ec8bd9bf22a2ecba4acf973925ef70553be67fc1220dcba08e not found: ID does not exist"
May 29 02:55:29.598179 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.598160    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 02:55:29.598469 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.598457    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerName="kube-rbac-proxy-thanos"
May 29 02:55:29.598514 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.598472    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerName="kube-rbac-proxy-thanos"
May 29 02:55:29.598514 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.598483    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerName="config-reloader"
May 29 02:55:29.598514 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.598489    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerName="config-reloader"
May 29 02:55:29.598514 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.598500    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerName="init-config-reloader"
May 29 02:55:29.598514 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.598507    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerName="init-config-reloader"
May 29 02:55:29.598514 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.598513    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerName="prometheus"
May 29 02:55:29.598686 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.598519    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerName="prometheus"
May 29 02:55:29.598686 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.598525    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerName="kube-rbac-proxy-web"
May 29 02:55:29.598686 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.598530    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerName="kube-rbac-proxy-web"
May 29 02:55:29.598686 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.598541    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerName="thanos-sidecar"
May 29 02:55:29.598686 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.598546    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerName="thanos-sidecar"
May 29 02:55:29.598686 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.598552    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerName="kube-rbac-proxy"
May 29 02:55:29.598686 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.598557    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerName="kube-rbac-proxy"
May 29 02:55:29.598686 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.598602    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerName="thanos-sidecar"
May 29 02:55:29.598686 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.598612    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerName="prometheus"
May 29 02:55:29.598686 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.598618    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerName="kube-rbac-proxy"
May 29 02:55:29.598686 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.598624    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerName="config-reloader"
May 29 02:55:29.598686 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.598630    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerName="kube-rbac-proxy-web"
May 29 02:55:29.598686 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.598635    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" containerName="kube-rbac-proxy-thanos"
May 29 02:55:29.604584 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.604564    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.607020 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.606999    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
May 29 02:55:29.607020 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.607014    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-qxtlb\""
May 29 02:55:29.607166 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.607075    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
May 29 02:55:29.607166 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.607135    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
May 29 02:55:29.607312 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.607296    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
May 29 02:55:29.607374 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.607361    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
May 29 02:55:29.607430 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.607360    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
May 29 02:55:29.607849 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.607834    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
May 29 02:55:29.608057 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.608042    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
May 29 02:55:29.608126 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.608113    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
May 29 02:55:29.608183 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.608167    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-f6mrds684j17m\""
May 29 02:55:29.608472 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.608451    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
May 29 02:55:29.608687 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.608664    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
May 29 02:55:29.610880 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.610861    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
May 29 02:55:29.613987 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.613948    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
May 29 02:55:29.615818 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.615799    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 02:55:29.710455 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.710418    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.710455 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.710460    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.710689 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.710486    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.710689 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.710510    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tlsr\" (UniqueName: \"kubernetes.io/projected/e73ab701-75d2-4f51-9e02-bb1145342d8a-kube-api-access-5tlsr\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.710689 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.710548    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e73ab701-75d2-4f51-9e02-bb1145342d8a-config-out\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.710689 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.710572    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.710689 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.710596    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-config\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.710689 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.710616    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e73ab701-75d2-4f51-9e02-bb1145342d8a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.710689 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.710640    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e73ab701-75d2-4f51-9e02-bb1145342d8a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.710689 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.710668    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e73ab701-75d2-4f51-9e02-bb1145342d8a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.711047 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.710695    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e73ab701-75d2-4f51-9e02-bb1145342d8a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.711047 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.710746    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.711047 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.710780    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e73ab701-75d2-4f51-9e02-bb1145342d8a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.711047 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.710814    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.711047 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.710838    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e73ab701-75d2-4f51-9e02-bb1145342d8a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.711047 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.710876    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e73ab701-75d2-4f51-9e02-bb1145342d8a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.711047 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.710931    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.711047 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.711005    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-web-config\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.811588 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.811517    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e73ab701-75d2-4f51-9e02-bb1145342d8a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.811588 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.811548    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.811588 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.811564    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e73ab701-75d2-4f51-9e02-bb1145342d8a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.812110 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.811593    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e73ab701-75d2-4f51-9e02-bb1145342d8a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.812110 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.811624    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.812110 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.811664    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-web-config\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.812110 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.811691    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.812110 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.811719    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.812110 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.811747    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.812110 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.811778    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5tlsr\" (UniqueName: \"kubernetes.io/projected/e73ab701-75d2-4f51-9e02-bb1145342d8a-kube-api-access-5tlsr\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.812110 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.811823    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e73ab701-75d2-4f51-9e02-bb1145342d8a-config-out\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.812110 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.811932    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e73ab701-75d2-4f51-9e02-bb1145342d8a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.812110 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.811944    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.812110 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.812011    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-config\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.812110 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.812040    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e73ab701-75d2-4f51-9e02-bb1145342d8a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.812656 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.812142    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e73ab701-75d2-4f51-9e02-bb1145342d8a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.812656 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.812177    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e73ab701-75d2-4f51-9e02-bb1145342d8a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.812656 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.812258    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e73ab701-75d2-4f51-9e02-bb1145342d8a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.812656 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.812312    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.812885 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.812865    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e73ab701-75d2-4f51-9e02-bb1145342d8a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.813987 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.813947    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e73ab701-75d2-4f51-9e02-bb1145342d8a-config-out\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.815263 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.815245    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
May 29 02:55:29.815810 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.815787    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
May 29 02:55:29.815945 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.815882    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
May 29 02:55:29.815945 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.815897    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
May 29 02:55:29.816106 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.815980    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
May 29 02:55:29.816162 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.816113    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
May 29 02:55:29.816314 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.816297    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
May 29 02:55:29.816628 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.816601    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
May 29 02:55:29.816762 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.816748    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
May 29 02:55:29.816951 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.816935    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-f6mrds684j17m\""
May 29 02:55:29.817053 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.816990    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
May 29 02:55:29.817117 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.816995    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
May 29 02:55:29.818114 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.818097    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
May 29 02:55:29.821703 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.821682    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
May 29 02:55:29.822564 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.822541    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e73ab701-75d2-4f51-9e02-bb1145342d8a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.822689 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.822668    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tlsr\" (UniqueName: \"kubernetes.io/projected/e73ab701-75d2-4f51-9e02-bb1145342d8a-kube-api-access-5tlsr\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.823794 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.823767    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e73ab701-75d2-4f51-9e02-bb1145342d8a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.824241 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.824215    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e73ab701-75d2-4f51-9e02-bb1145342d8a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.824382 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.824339    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.824502 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.824479    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.825076 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.825053    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.825314 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.825280    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e73ab701-75d2-4f51-9e02-bb1145342d8a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.826850 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.826822    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-web-config\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.826850 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.826841    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.827005 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.826851    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.827005 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.826913    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-config\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.827193 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.827160    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e73ab701-75d2-4f51-9e02-bb1145342d8a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.827663 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.827641    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.828132 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.828115    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e73ab701-75d2-4f51-9e02-bb1145342d8a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e73ab701-75d2-4f51-9e02-bb1145342d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:29.851222 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.851201    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e91e8e40-8028-423b-9fe2-8260065aa3c7" path="/var/lib/kubelet/pods/e91e8e40-8028-423b-9fe2-8260065aa3c7/volumes"
May 29 02:55:29.916613 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.916594    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-qxtlb\""
May 29 02:55:29.924518 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:29.924505    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:55:30.065157 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:30.065090    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 02:55:30.067124 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:55:30.067097    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode73ab701_75d2_4f51_9e02_bb1145342d8a.slice/crio-9226735b689ed8dc994b1a3524f919b83f0b5a1e86d31ff60041245b2e2e7bec WatchSource:0}: Error finding container 9226735b689ed8dc994b1a3524f919b83f0b5a1e86d31ff60041245b2e2e7bec: Status 404 returned error can't find the container with id 9226735b689ed8dc994b1a3524f919b83f0b5a1e86d31ff60041245b2e2e7bec
May 29 02:55:30.536799 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:30.536766    2559 generic.go:358] "Generic (PLEG): container finished" podID="e73ab701-75d2-4f51-9e02-bb1145342d8a" containerID="185fc16a0e276d598de16fc2d6b8d8c22971afd7b885237104bff20ee125b2b8" exitCode=0
May 29 02:55:30.536956 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:30.536854    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e73ab701-75d2-4f51-9e02-bb1145342d8a","Type":"ContainerDied","Data":"185fc16a0e276d598de16fc2d6b8d8c22971afd7b885237104bff20ee125b2b8"}
May 29 02:55:30.536956 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:30.536887    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e73ab701-75d2-4f51-9e02-bb1145342d8a","Type":"ContainerStarted","Data":"9226735b689ed8dc994b1a3524f919b83f0b5a1e86d31ff60041245b2e2e7bec"}
May 29 02:55:31.542779 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:31.542745    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e73ab701-75d2-4f51-9e02-bb1145342d8a","Type":"ContainerStarted","Data":"96954d99fa5e82645a4647c7da607fb1328629be4e33ab4032e7f146114ae26d"}
May 29 02:55:31.542779 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:31.542781    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e73ab701-75d2-4f51-9e02-bb1145342d8a","Type":"ContainerStarted","Data":"03160ae3c9948a43ba507609d04e49529b31876ef17ead265a0b6c896fceddd4"}
May 29 02:55:31.543192 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:31.542790    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e73ab701-75d2-4f51-9e02-bb1145342d8a","Type":"ContainerStarted","Data":"ceff47f32034ac69d19bc8393b41fd2e7156dd8b18324515312f4b1b0da66c26"}
May 29 02:55:31.543192 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:31.542799    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e73ab701-75d2-4f51-9e02-bb1145342d8a","Type":"ContainerStarted","Data":"cf46cf6d28da30339ff3a61379a43e265051d75ca1015408318b277c4904be38"}
May 29 02:55:31.543192 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:31.542808    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e73ab701-75d2-4f51-9e02-bb1145342d8a","Type":"ContainerStarted","Data":"d113dea90483d3f73bb524b5113e86f34b2415cad6c43ffc27c01f40f8607a00"}
May 29 02:55:31.543192 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:31.542815    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e73ab701-75d2-4f51-9e02-bb1145342d8a","Type":"ContainerStarted","Data":"860e5df1afd26fe743e81ec54def9abe05e6f1e486a47b57050e0c61f4fb9755"}
May 29 02:55:31.582357 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:31.582317    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.582304163 podStartE2EDuration="2.582304163s" podCreationTimestamp="2026-05-29 02:55:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 02:55:31.582230603 +0000 UTC m=+182.337545668" watchObservedRunningTime="2026-05-29 02:55:31.582304163 +0000 UTC m=+182.337619227"
May 29 02:55:34.924786 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:55:34.924750    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:56:29.925560 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:56:29.925530    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:56:29.940799 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:56:29.940776    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:56:30.727510 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:56:30.727484    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:57:29.759369 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:57:29.759343    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psj9n_d9a08de9-bea6-4561-8eae-62421b7584a9/ovn-acl-logging/0.log"
May 29 02:57:29.761935 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:57:29.761912    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psj9n_d9a08de9-bea6-4561-8eae-62421b7584a9/ovn-acl-logging/0.log"
May 29 02:57:29.762824 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:57:29.762810    2559 kubelet.go:1628] "Image garbage collection succeeded"
May 29 02:57:33.634194 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:57:33.634160    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-hp2fn"]
May 29 02:57:33.637287 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:57:33.637272    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp2fn"
May 29 02:57:33.639148 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:57:33.639133    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 29 02:57:33.646137 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:57:33.646120    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hp2fn"]
May 29 02:57:33.755541 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:57:33.755511    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/75bf9ff1-3a9f-4140-9ba2-bac8a5b588c1-kubelet-config\") pod \"global-pull-secret-syncer-hp2fn\" (UID: \"75bf9ff1-3a9f-4140-9ba2-bac8a5b588c1\") " pod="kube-system/global-pull-secret-syncer-hp2fn"
May 29 02:57:33.755654 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:57:33.755550    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/75bf9ff1-3a9f-4140-9ba2-bac8a5b588c1-dbus\") pod \"global-pull-secret-syncer-hp2fn\" (UID: \"75bf9ff1-3a9f-4140-9ba2-bac8a5b588c1\") " pod="kube-system/global-pull-secret-syncer-hp2fn"
May 29 02:57:33.755654 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:57:33.755574    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75bf9ff1-3a9f-4140-9ba2-bac8a5b588c1-original-pull-secret\") pod \"global-pull-secret-syncer-hp2fn\" (UID: \"75bf9ff1-3a9f-4140-9ba2-bac8a5b588c1\") " pod="kube-system/global-pull-secret-syncer-hp2fn"
May 29 02:57:33.855925 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:57:33.855899    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/75bf9ff1-3a9f-4140-9ba2-bac8a5b588c1-kubelet-config\") pod \"global-pull-secret-syncer-hp2fn\" (UID: \"75bf9ff1-3a9f-4140-9ba2-bac8a5b588c1\") " pod="kube-system/global-pull-secret-syncer-hp2fn"
May 29 02:57:33.856071 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:57:33.855939    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/75bf9ff1-3a9f-4140-9ba2-bac8a5b588c1-dbus\") pod \"global-pull-secret-syncer-hp2fn\" (UID: \"75bf9ff1-3a9f-4140-9ba2-bac8a5b588c1\") " pod="kube-system/global-pull-secret-syncer-hp2fn"
May 29 02:57:33.856071 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:57:33.855988    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75bf9ff1-3a9f-4140-9ba2-bac8a5b588c1-original-pull-secret\") pod \"global-pull-secret-syncer-hp2fn\" (UID: \"75bf9ff1-3a9f-4140-9ba2-bac8a5b588c1\") " pod="kube-system/global-pull-secret-syncer-hp2fn"
May 29 02:57:33.856071 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:57:33.856012    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/75bf9ff1-3a9f-4140-9ba2-bac8a5b588c1-kubelet-config\") pod \"global-pull-secret-syncer-hp2fn\" (UID: \"75bf9ff1-3a9f-4140-9ba2-bac8a5b588c1\") " pod="kube-system/global-pull-secret-syncer-hp2fn"
May 29 02:57:33.856193 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:57:33.856081    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/75bf9ff1-3a9f-4140-9ba2-bac8a5b588c1-dbus\") pod \"global-pull-secret-syncer-hp2fn\" (UID: \"75bf9ff1-3a9f-4140-9ba2-bac8a5b588c1\") " pod="kube-system/global-pull-secret-syncer-hp2fn"
May 29 02:57:33.858174 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:57:33.858149    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75bf9ff1-3a9f-4140-9ba2-bac8a5b588c1-original-pull-secret\") pod \"global-pull-secret-syncer-hp2fn\" (UID: \"75bf9ff1-3a9f-4140-9ba2-bac8a5b588c1\") " pod="kube-system/global-pull-secret-syncer-hp2fn"
May 29 02:57:33.946552 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:57:33.946483    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hp2fn"
May 29 02:57:34.067538 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:57:34.067462    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hp2fn"]
May 29 02:57:34.070252 ip-10-0-137-122 kubenswrapper[2559]: W0529 02:57:34.070227    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75bf9ff1_3a9f_4140_9ba2_bac8a5b588c1.slice/crio-770cab8c808f88d9abf7edaea287af91a51eb9b55f2c8da6293b368ac269a621 WatchSource:0}: Error finding container 770cab8c808f88d9abf7edaea287af91a51eb9b55f2c8da6293b368ac269a621: Status 404 returned error can't find the container with id 770cab8c808f88d9abf7edaea287af91a51eb9b55f2c8da6293b368ac269a621
May 29 02:57:34.071771 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:57:34.071754    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 29 02:57:34.884990 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:57:34.884945    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hp2fn" event={"ID":"75bf9ff1-3a9f-4140-9ba2-bac8a5b588c1","Type":"ContainerStarted","Data":"770cab8c808f88d9abf7edaea287af91a51eb9b55f2c8da6293b368ac269a621"}
May 29 02:57:37.895596 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:57:37.895563    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hp2fn" event={"ID":"75bf9ff1-3a9f-4140-9ba2-bac8a5b588c1","Type":"ContainerStarted","Data":"8096e03f8bbea0ac4bc44f2cf8949999a38756b21c57769c8a6fbfdb2301f91e"}
May 29 02:57:37.912606 ip-10-0-137-122 kubenswrapper[2559]: I0529 02:57:37.912560    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hp2fn" podStartSLOduration=1.4827833080000001 podStartE2EDuration="4.912543566s" podCreationTimestamp="2026-05-29 02:57:33 +0000 UTC" firstStartedPulling="2026-05-29 02:57:34.071907978 +0000 UTC m=+304.827223019" lastFinishedPulling="2026-05-29 02:57:37.501668235 +0000 UTC m=+308.256983277" observedRunningTime="2026-05-29 02:57:37.91132808 +0000 UTC m=+308.666643143" watchObservedRunningTime="2026-05-29 02:57:37.912543566 +0000 UTC m=+308.667858645"
May 29 03:02:29.786879 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:02:29.786840    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psj9n_d9a08de9-bea6-4561-8eae-62421b7584a9/ovn-acl-logging/0.log"
May 29 03:02:29.788308 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:02:29.788282    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psj9n_d9a08de9-bea6-4561-8eae-62421b7584a9/ovn-acl-logging/0.log"
May 29 03:04:55.592549 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:04:55.592515    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-854544c579-rsw9v"]
May 29 03:04:55.595727 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:04:55.595705    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-854544c579-rsw9v"
May 29 03:04:55.597792 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:04:55.597756    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 29 03:04:55.597792 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:04:55.597784    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-2jgj7\""
May 29 03:04:55.598125 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:04:55.598101    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 29 03:04:55.603650 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:04:55.603608    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-854544c579-rsw9v"]
May 29 03:04:55.649053 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:04:55.649021    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5cs5\" (UniqueName: \"kubernetes.io/projected/c9aa45f3-dad4-4428-a568-79512ff2849e-kube-api-access-r5cs5\") pod \"redis-854544c579-rsw9v\" (UID: \"c9aa45f3-dad4-4428-a568-79512ff2849e\") " pod="test-ns-feast/redis-854544c579-rsw9v"
May 29 03:04:55.750249 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:04:55.750219    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r5cs5\" (UniqueName: \"kubernetes.io/projected/c9aa45f3-dad4-4428-a568-79512ff2849e-kube-api-access-r5cs5\") pod \"redis-854544c579-rsw9v\" (UID: \"c9aa45f3-dad4-4428-a568-79512ff2849e\") " pod="test-ns-feast/redis-854544c579-rsw9v"
May 29 03:04:55.758555 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:04:55.758534    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5cs5\" (UniqueName: \"kubernetes.io/projected/c9aa45f3-dad4-4428-a568-79512ff2849e-kube-api-access-r5cs5\") pod \"redis-854544c579-rsw9v\" (UID: \"c9aa45f3-dad4-4428-a568-79512ff2849e\") " pod="test-ns-feast/redis-854544c579-rsw9v"
May 29 03:04:55.917180 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:04:55.917140    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-854544c579-rsw9v"
May 29 03:04:56.039088 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:04:56.039057    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-854544c579-rsw9v"]
May 29 03:04:56.041685 ip-10-0-137-122 kubenswrapper[2559]: W0529 03:04:56.041659    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc9aa45f3_dad4_4428_a568_79512ff2849e.slice/crio-ca5a2ca0a2f31d5a512c36553a4b6f77084bbf56771ada91101c12c7311c26f9 WatchSource:0}: Error finding container ca5a2ca0a2f31d5a512c36553a4b6f77084bbf56771ada91101c12c7311c26f9: Status 404 returned error can't find the container with id ca5a2ca0a2f31d5a512c36553a4b6f77084bbf56771ada91101c12c7311c26f9
May 29 03:04:56.043363 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:04:56.043345    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 29 03:04:56.111461 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:04:56.111428    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-854544c579-rsw9v" event={"ID":"c9aa45f3-dad4-4428-a568-79512ff2849e","Type":"ContainerStarted","Data":"ca5a2ca0a2f31d5a512c36553a4b6f77084bbf56771ada91101c12c7311c26f9"}
May 29 03:04:58.118492 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:04:58.118460    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-854544c579-rsw9v" event={"ID":"c9aa45f3-dad4-4428-a568-79512ff2849e","Type":"ContainerStarted","Data":"e048eea5ca02f7298678ef4e31f4d8e8a94337dfe63a41e0c3cee8f832ef769a"}
May 29 03:04:58.133594 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:04:58.133549    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/redis-854544c579-rsw9v" podStartSLOduration=1.495099326 podStartE2EDuration="3.133536252s" podCreationTimestamp="2026-05-29 03:04:55 +0000 UTC" firstStartedPulling="2026-05-29 03:04:56.043470427 +0000 UTC m=+746.798785469" lastFinishedPulling="2026-05-29 03:04:57.68190735 +0000 UTC m=+748.437222395" observedRunningTime="2026-05-29 03:04:58.131742314 +0000 UTC m=+748.887057378" watchObservedRunningTime="2026-05-29 03:04:58.133536252 +0000 UTC m=+748.888851315"
May 29 03:06:28.137988 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:06:28.137938    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-854544c579-rsw9v"]
May 29 03:06:28.138479 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:06:28.138149    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/redis-854544c579-rsw9v" podUID="c9aa45f3-dad4-4428-a568-79512ff2849e" containerName="redis" containerID="cri-o://e048eea5ca02f7298678ef4e31f4d8e8a94337dfe63a41e0c3cee8f832ef769a" gracePeriod=30
May 29 03:06:28.373659 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:06:28.373632    2559 generic.go:358] "Generic (PLEG): container finished" podID="c9aa45f3-dad4-4428-a568-79512ff2849e" containerID="e048eea5ca02f7298678ef4e31f4d8e8a94337dfe63a41e0c3cee8f832ef769a" exitCode=0
May 29 03:06:28.373774 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:06:28.373706    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-854544c579-rsw9v" event={"ID":"c9aa45f3-dad4-4428-a568-79512ff2849e","Type":"ContainerDied","Data":"e048eea5ca02f7298678ef4e31f4d8e8a94337dfe63a41e0c3cee8f832ef769a"}
May 29 03:06:28.373774 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:06:28.373739    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-854544c579-rsw9v" event={"ID":"c9aa45f3-dad4-4428-a568-79512ff2849e","Type":"ContainerDied","Data":"ca5a2ca0a2f31d5a512c36553a4b6f77084bbf56771ada91101c12c7311c26f9"}
May 29 03:06:28.373774 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:06:28.373748    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca5a2ca0a2f31d5a512c36553a4b6f77084bbf56771ada91101c12c7311c26f9"
May 29 03:06:28.379607 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:06:28.379590    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-854544c579-rsw9v"
May 29 03:06:28.545742 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:06:28.545665    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5cs5\" (UniqueName: \"kubernetes.io/projected/c9aa45f3-dad4-4428-a568-79512ff2849e-kube-api-access-r5cs5\") pod \"c9aa45f3-dad4-4428-a568-79512ff2849e\" (UID: \"c9aa45f3-dad4-4428-a568-79512ff2849e\") "
May 29 03:06:28.547676 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:06:28.547652    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9aa45f3-dad4-4428-a568-79512ff2849e-kube-api-access-r5cs5" (OuterVolumeSpecName: "kube-api-access-r5cs5") pod "c9aa45f3-dad4-4428-a568-79512ff2849e" (UID: "c9aa45f3-dad4-4428-a568-79512ff2849e"). InnerVolumeSpecName "kube-api-access-r5cs5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 03:06:28.646393 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:06:28.646364    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5cs5\" (UniqueName: \"kubernetes.io/projected/c9aa45f3-dad4-4428-a568-79512ff2849e-kube-api-access-r5cs5\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:06:29.376986 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:06:29.376938    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-854544c579-rsw9v"
May 29 03:06:29.395021 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:06:29.394994    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-854544c579-rsw9v"]
May 29 03:06:29.398472 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:06:29.398449    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-854544c579-rsw9v"]
May 29 03:06:29.848084 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:06:29.848002    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9aa45f3-dad4-4428-a568-79512ff2849e" path="/var/lib/kubelet/pods/c9aa45f3-dad4-4428-a568-79512ff2849e/volumes"
May 29 03:07:29.809639 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:07:29.809563    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psj9n_d9a08de9-bea6-4561-8eae-62421b7584a9/ovn-acl-logging/0.log"
May 29 03:07:29.809639 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:07:29.809584    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psj9n_d9a08de9-bea6-4561-8eae-62421b7584a9/ovn-acl-logging/0.log"
May 29 03:11:27.954745 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:27.954710    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"]
May 29 03:11:27.955294 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:27.955168    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9aa45f3-dad4-4428-a568-79512ff2849e" containerName="redis"
May 29 03:11:27.955294 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:27.955183    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9aa45f3-dad4-4428-a568-79512ff2849e" containerName="redis"
May 29 03:11:27.955294 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:27.955265    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="c9aa45f3-dad4-4428-a568-79512ff2849e" containerName="redis"
May 29 03:11:27.958941 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:27.958918    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:27.963247 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:27.963226    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 29 03:11:27.963247 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:27.963239    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 29 03:11:27.963453 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:27.963433    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 29 03:11:27.963541 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:27.963441    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-z5pl8\""
May 29 03:11:27.963617 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:27.963528    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 29 03:11:27.964204 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:27.964172    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 29 03:11:27.964695 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:27.964678    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 29 03:11:27.984633 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:27.984611    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"]
May 29 03:11:28.065511 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:28.065487    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbfmr\" (UniqueName: \"kubernetes.io/projected/d85faee4-e3bf-40f0-9654-2398f5f26c2e-kube-api-access-dbfmr\") pod \"feast-simple-feast-setup-664fcfcc58-s8jf6\" (UID: \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:28.065646 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:28.065518    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-s8jf6\" (UID: \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:28.065646 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:28.065562    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-s8jf6\" (UID: \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:28.065646 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:28.065589    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-s8jf6\" (UID: \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:28.065646 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:28.065606    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d85faee4-e3bf-40f0-9654-2398f5f26c2e-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-s8jf6\" (UID: \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:28.065788 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:28.065679    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-s8jf6\" (UID: \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:28.166596 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:28.166561    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dbfmr\" (UniqueName: \"kubernetes.io/projected/d85faee4-e3bf-40f0-9654-2398f5f26c2e-kube-api-access-dbfmr\") pod \"feast-simple-feast-setup-664fcfcc58-s8jf6\" (UID: \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:28.166746 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:28.166608    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-s8jf6\" (UID: \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:28.166746 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:28.166675    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-s8jf6\" (UID: \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:28.166746 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:28.166707    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-s8jf6\" (UID: \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:28.166746 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:28.166732    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d85faee4-e3bf-40f0-9654-2398f5f26c2e-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-s8jf6\" (UID: \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:28.166959 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:28.166773    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-s8jf6\" (UID: \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:28.166959 ip-10-0-137-122 kubenswrapper[2559]: E0529 03:11:28.166774    2559 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found
May 29 03:11:28.166959 ip-10-0-137-122 kubenswrapper[2559]: E0529 03:11:28.166885    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-registry-tls podName:d85faee4-e3bf-40f0-9654-2398f5f26c2e nodeName:}" failed. No retries permitted until 2026-05-29 03:11:28.666863921 +0000 UTC m=+1139.422178972 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-registry-tls") pod "feast-simple-feast-setup-664fcfcc58-s8jf6" (UID: "d85faee4-e3bf-40f0-9654-2398f5f26c2e") : secret "feast-simple-feast-setup-registry-tls" not found
May 29 03:11:28.167200 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:28.167172    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d85faee4-e3bf-40f0-9654-2398f5f26c2e-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-s8jf6\" (UID: \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:28.169105 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:28.169082    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-s8jf6\" (UID: \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:28.169197 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:28.169118    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-s8jf6\" (UID: \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:28.169197 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:28.169126    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-s8jf6\" (UID: \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:28.177251 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:28.177229    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbfmr\" (UniqueName: \"kubernetes.io/projected/d85faee4-e3bf-40f0-9654-2398f5f26c2e-kube-api-access-dbfmr\") pod \"feast-simple-feast-setup-664fcfcc58-s8jf6\" (UID: \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:28.671693 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:28.671661    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-s8jf6\" (UID: \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:28.673956 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:28.673932    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-s8jf6\" (UID: \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:28.879891 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:28.879863    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:29.009993 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:29.009944    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"]
May 29 03:11:29.011951 ip-10-0-137-122 kubenswrapper[2559]: W0529 03:11:29.011924    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd85faee4_e3bf_40f0_9654_2398f5f26c2e.slice/crio-8a343daa6ae52f97121a0ebc49d9581d60e80e6beaf0bada070153ee4c7c37b9 WatchSource:0}: Error finding container 8a343daa6ae52f97121a0ebc49d9581d60e80e6beaf0bada070153ee4c7c37b9: Status 404 returned error can't find the container with id 8a343daa6ae52f97121a0ebc49d9581d60e80e6beaf0bada070153ee4c7c37b9
May 29 03:11:29.013607 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:29.013590    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 29 03:11:29.218498 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:29.218421    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" event={"ID":"d85faee4-e3bf-40f0-9654-2398f5f26c2e","Type":"ContainerStarted","Data":"8a343daa6ae52f97121a0ebc49d9581d60e80e6beaf0bada070153ee4c7c37b9"}
May 29 03:11:29.821029 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:29.820983    2559 scope.go:117] "RemoveContainer" containerID="e048eea5ca02f7298678ef4e31f4d8e8a94337dfe63a41e0c3cee8f832ef769a"
May 29 03:11:44.274080 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:44.274044    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" event={"ID":"d85faee4-e3bf-40f0-9654-2398f5f26c2e","Type":"ContainerStarted","Data":"1164603e6a6f611525beadd252fc4a751af636b5663b1f748cde92a2a6b4c53b"}
May 29 03:11:48.287336 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:48.287301    2559 generic.go:358] "Generic (PLEG): container finished" podID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerID="1164603e6a6f611525beadd252fc4a751af636b5663b1f748cde92a2a6b4c53b" exitCode=0
May 29 03:11:48.287803 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:48.287340    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" event={"ID":"d85faee4-e3bf-40f0-9654-2398f5f26c2e","Type":"ContainerDied","Data":"1164603e6a6f611525beadd252fc4a751af636b5663b1f748cde92a2a6b4c53b"}
May 29 03:11:49.291850 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:49.291767    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" event={"ID":"d85faee4-e3bf-40f0-9654-2398f5f26c2e","Type":"ContainerStarted","Data":"3ba83fed1829ba721c1af39e95411f57be097827b0c12949efa0deb719b22c8b"}
May 29 03:11:54.308081 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:54.308048    2559 generic.go:358] "Generic (PLEG): container finished" podID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerID="3ba83fed1829ba721c1af39e95411f57be097827b0c12949efa0deb719b22c8b" exitCode=0
May 29 03:11:54.308555 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:54.308124    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" event={"ID":"d85faee4-e3bf-40f0-9654-2398f5f26c2e","Type":"ContainerDied","Data":"3ba83fed1829ba721c1af39e95411f57be097827b0c12949efa0deb719b22c8b"}
May 29 03:11:55.315817 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:55.315774    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" event={"ID":"d85faee4-e3bf-40f0-9654-2398f5f26c2e","Type":"ContainerStarted","Data":"44bc8efc7ee1b0e17c4d0a7210c3289951497fa97206548069251016fbe677d7"}
May 29 03:11:55.316326 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:55.315828    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" event={"ID":"d85faee4-e3bf-40f0-9654-2398f5f26c2e","Type":"ContainerStarted","Data":"90ff5ea42e8a4f297618864469057dfde1443c8c0083ef7ac7ffc89a5f74bc22"}
May 29 03:11:55.316326 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:55.315845    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" event={"ID":"d85faee4-e3bf-40f0-9654-2398f5f26c2e","Type":"ContainerStarted","Data":"e26a4be08db3c9e1e8dc4ead8823f840455d2343ddc9d220a299497f1a821f7b"}
May 29 03:11:55.316326 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:55.315858    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" event={"ID":"d85faee4-e3bf-40f0-9654-2398f5f26c2e","Type":"ContainerStarted","Data":"3bdf24b24e6b215620cd3bf1fadecf120327a850198811981b06c85a57cae9c9"}
May 29 03:11:55.342086 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:55.341990    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" podStartSLOduration=14.068504446 podStartE2EDuration="28.341946362s" podCreationTimestamp="2026-05-29 03:11:27 +0000 UTC" firstStartedPulling="2026-05-29 03:11:29.013717389 +0000 UTC m=+1139.769032431" lastFinishedPulling="2026-05-29 03:11:43.287159292 +0000 UTC m=+1154.042474347" observedRunningTime="2026-05-29 03:11:55.339605557 +0000 UTC m=+1166.094920623" watchObservedRunningTime="2026-05-29 03:11:55.341946362 +0000 UTC m=+1166.097261427"
May 29 03:11:55.880950 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:55.880911    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:55.880950 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:55.880956    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:55.881225 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:55.880987    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:55.881225 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:55.881001    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:55.883099 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:55.883055    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused"
May 29 03:11:55.883334 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:55.883079    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused"
May 29 03:11:55.883422 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:55.883090    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.23:6571: connect: connection refused"
May 29 03:11:55.883422 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:55.883145    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.23:8816: connect: connection refused"
May 29 03:11:58.880188 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:58.880142    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:58.880782 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:58.880200    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:58.880782 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:58.880484    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused"
May 29 03:11:58.880782 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:58.880485    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.23:8816: connect: connection refused"
May 29 03:11:58.880782 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:58.880765    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused"
May 29 03:11:58.881099 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:58.880906    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:58.881099 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:58.880930    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:11:58.881343 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:11:58.881313    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.23:6571: connect: connection refused"
May 29 03:12:01.881412 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:12:01.881383    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:12:01.881894 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:12:01.881869    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:12:01.882068 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:12:01.882055    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:12:01.882140 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:12:01.882120    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:12:01.886003 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:12:01.885987    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:12:02.339900 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:12:02.339823    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:12:02.339900 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:12:02.339875    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:12:02.342222 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:12:02.342208    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:12:29.832428 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:12:29.832401    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psj9n_d9a08de9-bea6-4561-8eae-62421b7584a9/ovn-acl-logging/0.log"
May 29 03:12:29.833411 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:12:29.833388    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psj9n_d9a08de9-bea6-4561-8eae-62421b7584a9/ovn-acl-logging/0.log"
May 29 03:13:05.185755 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.185720    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"]
May 29 03:13:05.186250 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.186186    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="registry" containerID="cri-o://3bdf24b24e6b215620cd3bf1fadecf120327a850198811981b06c85a57cae9c9" gracePeriod=30
May 29 03:13:05.186322 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.186266    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="offline" containerID="cri-o://90ff5ea42e8a4f297618864469057dfde1443c8c0083ef7ac7ffc89a5f74bc22" gracePeriod=30
May 29 03:13:05.186322 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.186269    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="ui" containerID="cri-o://44bc8efc7ee1b0e17c4d0a7210c3289951497fa97206548069251016fbe677d7" gracePeriod=30
May 29 03:13:05.186418 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.186291    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="online" containerID="cri-o://e26a4be08db3c9e1e8dc4ead8823f840455d2343ddc9d220a299497f1a821f7b" gracePeriod=30
May 29 03:13:05.617801 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.617770    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"]
May 29 03:13:05.643413 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.642381    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"]
May 29 03:13:05.643413 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.642546    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:05.648017 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.645002    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-vm7v8\""
May 29 03:13:05.734593 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.734559    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/33a9650c-df7c-4082-b06b-ac0c33850488-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zk95v\" (UID: \"33a9650c-df7c-4082-b06b-ac0c33850488\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:05.734593 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.734594    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/33a9650c-df7c-4082-b06b-ac0c33850488-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-zk95v\" (UID: \"33a9650c-df7c-4082-b06b-ac0c33850488\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:05.734815 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.734622    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/33a9650c-df7c-4082-b06b-ac0c33850488-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zk95v\" (UID: \"33a9650c-df7c-4082-b06b-ac0c33850488\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:05.734815 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.734690    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/33a9650c-df7c-4082-b06b-ac0c33850488-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zk95v\" (UID: \"33a9650c-df7c-4082-b06b-ac0c33850488\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:05.734815 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.734762    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqxks\" (UniqueName: \"kubernetes.io/projected/33a9650c-df7c-4082-b06b-ac0c33850488-kube-api-access-pqxks\") pod \"feast-simple-feast-setup-664fcfcc58-zk95v\" (UID: \"33a9650c-df7c-4082-b06b-ac0c33850488\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:05.734815 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.734790    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/33a9650c-df7c-4082-b06b-ac0c33850488-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zk95v\" (UID: \"33a9650c-df7c-4082-b06b-ac0c33850488\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:05.836268 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.836232    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pqxks\" (UniqueName: \"kubernetes.io/projected/33a9650c-df7c-4082-b06b-ac0c33850488-kube-api-access-pqxks\") pod \"feast-simple-feast-setup-664fcfcc58-zk95v\" (UID: \"33a9650c-df7c-4082-b06b-ac0c33850488\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:05.836465 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.836283    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/33a9650c-df7c-4082-b06b-ac0c33850488-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zk95v\" (UID: \"33a9650c-df7c-4082-b06b-ac0c33850488\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:05.836465 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.836333    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/33a9650c-df7c-4082-b06b-ac0c33850488-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zk95v\" (UID: \"33a9650c-df7c-4082-b06b-ac0c33850488\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:05.836465 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.836359    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/33a9650c-df7c-4082-b06b-ac0c33850488-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-zk95v\" (UID: \"33a9650c-df7c-4082-b06b-ac0c33850488\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:05.836465 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.836400    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/33a9650c-df7c-4082-b06b-ac0c33850488-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zk95v\" (UID: \"33a9650c-df7c-4082-b06b-ac0c33850488\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:05.836465 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.836428    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/33a9650c-df7c-4082-b06b-ac0c33850488-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zk95v\" (UID: \"33a9650c-df7c-4082-b06b-ac0c33850488\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:05.836917 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.836886    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/33a9650c-df7c-4082-b06b-ac0c33850488-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-zk95v\" (UID: \"33a9650c-df7c-4082-b06b-ac0c33850488\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:05.839064 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.839039    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/33a9650c-df7c-4082-b06b-ac0c33850488-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zk95v\" (UID: \"33a9650c-df7c-4082-b06b-ac0c33850488\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:05.839193 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.839170    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/33a9650c-df7c-4082-b06b-ac0c33850488-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zk95v\" (UID: \"33a9650c-df7c-4082-b06b-ac0c33850488\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:05.839193 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.839177    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/33a9650c-df7c-4082-b06b-ac0c33850488-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zk95v\" (UID: \"33a9650c-df7c-4082-b06b-ac0c33850488\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:05.839625 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.839604    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/33a9650c-df7c-4082-b06b-ac0c33850488-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zk95v\" (UID: \"33a9650c-df7c-4082-b06b-ac0c33850488\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:05.848701 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.848680    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqxks\" (UniqueName: \"kubernetes.io/projected/33a9650c-df7c-4082-b06b-ac0c33850488-kube-api-access-pqxks\") pod \"feast-simple-feast-setup-664fcfcc58-zk95v\" (UID: \"33a9650c-df7c-4082-b06b-ac0c33850488\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:05.966955 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:05.966919    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:06.098175 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:06.098144    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"]
May 29 03:13:06.101396 ip-10-0-137-122 kubenswrapper[2559]: W0529 03:13:06.101363    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33a9650c_df7c_4082_b06b_ac0c33850488.slice/crio-20140be46d4219c2f97133f622b21392e9df930fdc0b50e5343029783e26737c WatchSource:0}: Error finding container 20140be46d4219c2f97133f622b21392e9df930fdc0b50e5343029783e26737c: Status 404 returned error can't find the container with id 20140be46d4219c2f97133f622b21392e9df930fdc0b50e5343029783e26737c
May 29 03:13:06.531836 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:06.531799    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" event={"ID":"33a9650c-df7c-4082-b06b-ac0c33850488","Type":"ContainerStarted","Data":"5f9129dfbe9ed58999a25da4d45be7ee9adb33898553f0c840541d3f9423e8d2"}
May 29 03:13:06.532327 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:06.531843    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" event={"ID":"33a9650c-df7c-4082-b06b-ac0c33850488","Type":"ContainerStarted","Data":"20140be46d4219c2f97133f622b21392e9df930fdc0b50e5343029783e26737c"}
May 29 03:13:06.534392 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:06.534368    2559 generic.go:358] "Generic (PLEG): container finished" podID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerID="44bc8efc7ee1b0e17c4d0a7210c3289951497fa97206548069251016fbe677d7" exitCode=0
May 29 03:13:06.534392 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:06.534390    2559 generic.go:358] "Generic (PLEG): container finished" podID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerID="e26a4be08db3c9e1e8dc4ead8823f840455d2343ddc9d220a299497f1a821f7b" exitCode=0
May 29 03:13:06.534549 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:06.534438    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" event={"ID":"d85faee4-e3bf-40f0-9654-2398f5f26c2e","Type":"ContainerDied","Data":"44bc8efc7ee1b0e17c4d0a7210c3289951497fa97206548069251016fbe677d7"}
May 29 03:13:06.534549 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:06.534473    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" event={"ID":"d85faee4-e3bf-40f0-9654-2398f5f26c2e","Type":"ContainerDied","Data":"e26a4be08db3c9e1e8dc4ead8823f840455d2343ddc9d220a299497f1a821f7b"}
May 29 03:13:11.549028 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:11.548995    2559 generic.go:358] "Generic (PLEG): container finished" podID="33a9650c-df7c-4082-b06b-ac0c33850488" containerID="5f9129dfbe9ed58999a25da4d45be7ee9adb33898553f0c840541d3f9423e8d2" exitCode=0
May 29 03:13:11.549426 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:11.549053    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" event={"ID":"33a9650c-df7c-4082-b06b-ac0c33850488","Type":"ContainerDied","Data":"5f9129dfbe9ed58999a25da4d45be7ee9adb33898553f0c840541d3f9423e8d2"}
May 29 03:13:11.882845 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:11.882809    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused"
May 29 03:13:12.340069 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:12.339937    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused"
May 29 03:13:12.553591 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:12.553560    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" event={"ID":"33a9650c-df7c-4082-b06b-ac0c33850488","Type":"ContainerStarted","Data":"accc65cfa4b91b525de66c051300a7c7c6d4db5552798f29ed97d4f5153fbb51"}
May 29 03:13:17.568653 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:17.568620    2559 generic.go:358] "Generic (PLEG): container finished" podID="33a9650c-df7c-4082-b06b-ac0c33850488" containerID="accc65cfa4b91b525de66c051300a7c7c6d4db5552798f29ed97d4f5153fbb51" exitCode=0
May 29 03:13:17.568653 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:17.568656    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" event={"ID":"33a9650c-df7c-4082-b06b-ac0c33850488","Type":"ContainerDied","Data":"accc65cfa4b91b525de66c051300a7c7c6d4db5552798f29ed97d4f5153fbb51"}
May 29 03:13:18.576583 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:18.576537    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" event={"ID":"33a9650c-df7c-4082-b06b-ac0c33850488","Type":"ContainerStarted","Data":"87825e635f99b6112d50390a1f8d6011e0a5500bfc794bd9b616456267515655"}
May 29 03:13:18.577075 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:18.576594    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" event={"ID":"33a9650c-df7c-4082-b06b-ac0c33850488","Type":"ContainerStarted","Data":"c4d75f3d609b7654fb6b5e1c5b1f6af6c12aab44ea5479205f80051ac890eeed"}
May 29 03:13:18.577075 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:18.576610    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" event={"ID":"33a9650c-df7c-4082-b06b-ac0c33850488","Type":"ContainerStarted","Data":"be6cac882c6a640588d9e60c5cd7cdc68951fd698175ee554e159a5f5809ee4f"}
May 29 03:13:18.577075 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:18.576631    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" event={"ID":"33a9650c-df7c-4082-b06b-ac0c33850488","Type":"ContainerStarted","Data":"4a845d76fd10290a925d836f8a1298dc1744140028a4c2030ed02c5a896ed4a9"}
May 29 03:13:18.605500 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:18.605434    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" podStartSLOduration=13.605417467 podStartE2EDuration="13.605417467s" podCreationTimestamp="2026-05-29 03:13:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 03:13:18.604242151 +0000 UTC m=+1249.359557240" watchObservedRunningTime="2026-05-29 03:13:18.605417467 +0000 UTC m=+1249.360732533"
May 29 03:13:20.967553 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:20.967385    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:20.967553 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:20.967436    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:20.967553 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:20.967451    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:20.967553 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:20.967463    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:20.969553 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:20.969515    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused"
May 29 03:13:20.969688 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:20.969563    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.24:8816: connect: connection refused"
May 29 03:13:20.969688 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:20.969522    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.24:6571: connect: connection refused"
May 29 03:13:20.969688 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:20.969525    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused"
May 29 03:13:21.882898 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:21.882840    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused"
May 29 03:13:22.340529 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:22.340435    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused"
May 29 03:13:23.968336 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:23.968302    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:23.968740 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:23.968489    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:23.968740 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:23.968595    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:23.969020 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:23.968994    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:23.969206 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:23.969056    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:23.969206 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:23.969073    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:23.969206 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:23.969133    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:23.969206 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:23.969179    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:23.969392 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:23.969376    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:23.972846 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:23.972829    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:24.599212 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:24.599182    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:24.602117 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:24.602098    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:13:31.882019 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:31.881953    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused"
May 29 03:13:31.882427 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:31.882131    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:13:32.340527 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:32.340425    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused"
May 29 03:13:32.340700 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:32.340554    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:13:35.637303 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:35.637271    2559 generic.go:358] "Generic (PLEG): container finished" podID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerID="90ff5ea42e8a4f297618864469057dfde1443c8c0083ef7ac7ffc89a5f74bc22" exitCode=137
May 29 03:13:35.637303 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:35.637300    2559 generic.go:358] "Generic (PLEG): container finished" podID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerID="3bdf24b24e6b215620cd3bf1fadecf120327a850198811981b06c85a57cae9c9" exitCode=137
May 29 03:13:35.637696 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:35.637328    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" event={"ID":"d85faee4-e3bf-40f0-9654-2398f5f26c2e","Type":"ContainerDied","Data":"90ff5ea42e8a4f297618864469057dfde1443c8c0083ef7ac7ffc89a5f74bc22"}
May 29 03:13:35.637696 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:35.637351    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" event={"ID":"d85faee4-e3bf-40f0-9654-2398f5f26c2e","Type":"ContainerDied","Data":"3bdf24b24e6b215620cd3bf1fadecf120327a850198811981b06c85a57cae9c9"}
May 29 03:13:35.831719 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:35.831697    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:13:35.988152 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:35.988063    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-offline-tls\") pod \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\" (UID: \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\") "
May 29 03:13:35.988152 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:35.988093    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-registry-tls\") pod \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\" (UID: \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\") "
May 29 03:13:35.988152 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:35.988131    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-ui-tls\") pod \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\" (UID: \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\") "
May 29 03:13:35.988393 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:35.988208    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-online-tls\") pod \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\" (UID: \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\") "
May 29 03:13:35.988393 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:35.988232    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d85faee4-e3bf-40f0-9654-2398f5f26c2e-feast-data\") pod \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\" (UID: \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\") "
May 29 03:13:35.988393 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:35.988260    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbfmr\" (UniqueName: \"kubernetes.io/projected/d85faee4-e3bf-40f0-9654-2398f5f26c2e-kube-api-access-dbfmr\") pod \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\" (UID: \"d85faee4-e3bf-40f0-9654-2398f5f26c2e\") "
May 29 03:13:35.988849 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:35.988807    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d85faee4-e3bf-40f0-9654-2398f5f26c2e-feast-data" (OuterVolumeSpecName: "feast-data") pod "d85faee4-e3bf-40f0-9654-2398f5f26c2e" (UID: "d85faee4-e3bf-40f0-9654-2398f5f26c2e"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 03:13:35.990518 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:35.990488    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "d85faee4-e3bf-40f0-9654-2398f5f26c2e" (UID: "d85faee4-e3bf-40f0-9654-2398f5f26c2e"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 03:13:35.990726 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:35.990700    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "d85faee4-e3bf-40f0-9654-2398f5f26c2e" (UID: "d85faee4-e3bf-40f0-9654-2398f5f26c2e"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 03:13:35.990726 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:35.990717    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d85faee4-e3bf-40f0-9654-2398f5f26c2e" (UID: "d85faee4-e3bf-40f0-9654-2398f5f26c2e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 03:13:35.990859 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:35.990803    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-online-tls" (OuterVolumeSpecName: "online-tls") pod "d85faee4-e3bf-40f0-9654-2398f5f26c2e" (UID: "d85faee4-e3bf-40f0-9654-2398f5f26c2e"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 03:13:35.990859 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:35.990814    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d85faee4-e3bf-40f0-9654-2398f5f26c2e-kube-api-access-dbfmr" (OuterVolumeSpecName: "kube-api-access-dbfmr") pod "d85faee4-e3bf-40f0-9654-2398f5f26c2e" (UID: "d85faee4-e3bf-40f0-9654-2398f5f26c2e"). InnerVolumeSpecName "kube-api-access-dbfmr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 03:13:36.088938 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:36.088911    2559 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-online-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:13:36.088938 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:36.088933    2559 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d85faee4-e3bf-40f0-9654-2398f5f26c2e-feast-data\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:13:36.089111 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:36.088943    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbfmr\" (UniqueName: \"kubernetes.io/projected/d85faee4-e3bf-40f0-9654-2398f5f26c2e-kube-api-access-dbfmr\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:13:36.089111 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:36.088953    2559 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-offline-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:13:36.089111 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:36.088962    2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-registry-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:13:36.089111 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:36.088991    2559 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d85faee4-e3bf-40f0-9654-2398f5f26c2e-ui-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:13:36.642828 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:36.642793    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6" event={"ID":"d85faee4-e3bf-40f0-9654-2398f5f26c2e","Type":"ContainerDied","Data":"8a343daa6ae52f97121a0ebc49d9581d60e80e6beaf0bada070153ee4c7c37b9"}
May 29 03:13:36.643260 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:36.642840    2559 scope.go:117] "RemoveContainer" containerID="44bc8efc7ee1b0e17c4d0a7210c3289951497fa97206548069251016fbe677d7"
May 29 03:13:36.643260 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:36.642801    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"
May 29 03:13:36.651334 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:36.651316    2559 scope.go:117] "RemoveContainer" containerID="90ff5ea42e8a4f297618864469057dfde1443c8c0083ef7ac7ffc89a5f74bc22"
May 29 03:13:36.658072 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:36.658057    2559 scope.go:117] "RemoveContainer" containerID="e26a4be08db3c9e1e8dc4ead8823f840455d2343ddc9d220a299497f1a821f7b"
May 29 03:13:36.665421 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:36.665348    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"]
May 29 03:13:36.665473 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:36.665446    2559 scope.go:117] "RemoveContainer" containerID="3bdf24b24e6b215620cd3bf1fadecf120327a850198811981b06c85a57cae9c9"
May 29 03:13:36.669068 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:36.669046    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-s8jf6"]
May 29 03:13:36.672303 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:36.672289    2559 scope.go:117] "RemoveContainer" containerID="3ba83fed1829ba721c1af39e95411f57be097827b0c12949efa0deb719b22c8b"
May 29 03:13:36.678790 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:36.678773    2559 scope.go:117] "RemoveContainer" containerID="1164603e6a6f611525beadd252fc4a751af636b5663b1f748cde92a2a6b4c53b"
May 29 03:13:37.848781 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:13:37.848749    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" path="/var/lib/kubelet/pods/d85faee4-e3bf-40f0-9654-2398f5f26c2e/volumes"
May 29 03:14:27.320352 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.320311    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"]
May 29 03:14:27.322002 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.320784    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="online"
May 29 03:14:27.322002 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.320799    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="online"
May 29 03:14:27.322002 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.320811    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="feast-apply"
May 29 03:14:27.322002 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.320820    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="feast-apply"
May 29 03:14:27.322002 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.320837    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="feast-init"
May 29 03:14:27.322002 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.320846    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="feast-init"
May 29 03:14:27.322002 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.320862    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="registry"
May 29 03:14:27.322002 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.320869    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="registry"
May 29 03:14:27.322002 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.320878    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="offline"
May 29 03:14:27.322002 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.320885    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="offline"
May 29 03:14:27.322002 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.320901    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="ui"
May 29 03:14:27.322002 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.320909    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="ui"
May 29 03:14:27.322002 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.321003    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="ui"
May 29 03:14:27.322002 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.321017    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="registry"
May 29 03:14:27.322002 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.321030    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="online"
May 29 03:14:27.322002 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.321040    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="d85faee4-e3bf-40f0-9654-2398f5f26c2e" containerName="offline"
May 29 03:14:27.323706 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.323689    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:27.326005 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.325980    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 29 03:14:27.326158 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.326137    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-4drjf\""
May 29 03:14:27.326311 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.326015    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 29 03:14:27.326374 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.326040    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 29 03:14:27.326374 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.326189    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 29 03:14:27.326476 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.326012    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 29 03:14:27.326476 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.326246    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 29 03:14:27.332921 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.332301    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"]
May 29 03:14:27.481053 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.481021    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-nn2z4\" (UID: \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:27.481053 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.481054    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-nn2z4\" (UID: \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:27.481232 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.481119    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-nn2z4\" (UID: \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:27.481232 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.481190    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-nn2z4\" (UID: \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:27.481232 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.481212    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-nn2z4\" (UID: \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:27.481334 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.481247    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84jxs\" (UniqueName: \"kubernetes.io/projected/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-kube-api-access-84jxs\") pod \"feast-simple-feast-remote-setup-5cff94746b-nn2z4\" (UID: \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:27.582592 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.582521    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-nn2z4\" (UID: \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:27.582592 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.582561    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-nn2z4\" (UID: \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:27.582774 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.582594    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-nn2z4\" (UID: \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:27.582774 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.582633    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-84jxs\" (UniqueName: \"kubernetes.io/projected/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-kube-api-access-84jxs\") pod \"feast-simple-feast-remote-setup-5cff94746b-nn2z4\" (UID: \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:27.582774 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.582679    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-nn2z4\" (UID: \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:27.582933 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.582783    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-nn2z4\" (UID: \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:27.583200 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.583167    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-nn2z4\" (UID: \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:27.583410 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.583388    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-nn2z4\" (UID: \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:27.585010 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.584992    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-nn2z4\" (UID: \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:27.585447 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.585429    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-nn2z4\" (UID: \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:27.585576 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.585558    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-nn2z4\" (UID: \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:27.590056 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.590036    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84jxs\" (UniqueName: \"kubernetes.io/projected/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-kube-api-access-84jxs\") pod \"feast-simple-feast-remote-setup-5cff94746b-nn2z4\" (UID: \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:27.636104 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.636074    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:27.764118 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.764093    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"]
May 29 03:14:27.790354 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:27.790329    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4" event={"ID":"4b928ca2-f0bb-41d2-9e34-1f6f2f787449","Type":"ContainerStarted","Data":"1d6d03252e1129de41bccb52b1281a98b00e4ae762dd7e238c1242fc0f0e3797"}
May 29 03:14:28.794931 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:28.794896    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4" event={"ID":"4b928ca2-f0bb-41d2-9e34-1f6f2f787449","Type":"ContainerStarted","Data":"bb2386ca2c05a00997078dd79f289a812d1079b97c88a5873df4bc442931b3d8"}
May 29 03:14:32.807344 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:32.807312    2559 generic.go:358] "Generic (PLEG): container finished" podID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerID="bb2386ca2c05a00997078dd79f289a812d1079b97c88a5873df4bc442931b3d8" exitCode=0
May 29 03:14:32.807718 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:32.807382    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4" event={"ID":"4b928ca2-f0bb-41d2-9e34-1f6f2f787449","Type":"ContainerDied","Data":"bb2386ca2c05a00997078dd79f289a812d1079b97c88a5873df4bc442931b3d8"}
May 29 03:14:33.812118 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:33.812082    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4" event={"ID":"4b928ca2-f0bb-41d2-9e34-1f6f2f787449","Type":"ContainerStarted","Data":"420554f1e6118c2dac9459193a508f34a7e08ce66f8064c9f65fd01cfb70f739"}
May 29 03:14:38.828496 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:38.828461    2559 generic.go:358] "Generic (PLEG): container finished" podID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerID="420554f1e6118c2dac9459193a508f34a7e08ce66f8064c9f65fd01cfb70f739" exitCode=0
May 29 03:14:38.828856 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:38.828535    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4" event={"ID":"4b928ca2-f0bb-41d2-9e34-1f6f2f787449","Type":"ContainerDied","Data":"420554f1e6118c2dac9459193a508f34a7e08ce66f8064c9f65fd01cfb70f739"}
May 29 03:14:39.835570 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:39.835527    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4" event={"ID":"4b928ca2-f0bb-41d2-9e34-1f6f2f787449","Type":"ContainerStarted","Data":"351d08111ac244cb515fd91c01d77d19c39b9d3a763749528a13076229dd8f90"}
May 29 03:14:39.835570 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:39.835574    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4" event={"ID":"4b928ca2-f0bb-41d2-9e34-1f6f2f787449","Type":"ContainerStarted","Data":"c0e21fdc829e525a9e97631e8176d0809b3463c0ec148a05d3532fc542ad52e8"}
May 29 03:14:39.836072 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:39.835590    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4" event={"ID":"4b928ca2-f0bb-41d2-9e34-1f6f2f787449","Type":"ContainerStarted","Data":"532af0022c192883f47649d7b9721d26d25272b71e99be118d862ec73dab9a72"}
May 29 03:14:39.860304 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:39.860238    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4" podStartSLOduration=12.860220828 podStartE2EDuration="12.860220828s" podCreationTimestamp="2026-05-29 03:14:27 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 03:14:39.858174901 +0000 UTC m=+1330.613489967" watchObservedRunningTime="2026-05-29 03:14:39.860220828 +0000 UTC m=+1330.615535892"
May 29 03:14:42.636185 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:42.636140    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:42.636641 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:42.636197    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:42.636894 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:42.636860    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:42.637643 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:42.637604    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4" podUID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.25:8816: connect: connection refused"
May 29 03:14:42.637643 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:42.637626    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4" podUID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused"
May 29 03:14:42.637790 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:42.637643    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4" podUID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused"
May 29 03:14:45.638142 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:45.638112    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:45.638503 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:45.638226    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:45.638554 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:45.638545    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:45.638592 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:45.638563    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:45.638999 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:45.638956    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:45.639091 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:45.639055    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:45.642299 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:45.642281    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:45.856726 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:45.856694    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:14:45.859652 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:14:45.859631    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:15:48.635519 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:15:48.635477    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"]
May 29 03:15:48.636069 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:15:48.635907    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4" podUID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerName="online" containerID="cri-o://532af0022c192883f47649d7b9721d26d25272b71e99be118d862ec73dab9a72" gracePeriod=30
May 29 03:15:48.636069 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:15:48.635927    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4" podUID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerName="ui" containerID="cri-o://351d08111ac244cb515fd91c01d77d19c39b9d3a763749528a13076229dd8f90" gracePeriod=30
May 29 03:15:48.636069 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:15:48.635984    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4" podUID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerName="offline" containerID="cri-o://c0e21fdc829e525a9e97631e8176d0809b3463c0ec148a05d3532fc542ad52e8" gracePeriod=30
May 29 03:15:48.777164 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:15:48.777129    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"]
May 29 03:15:48.777576 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:15:48.777520    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="registry" containerID="cri-o://4a845d76fd10290a925d836f8a1298dc1744140028a4c2030ed02c5a896ed4a9" gracePeriod=30
May 29 03:15:48.777713 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:15:48.777617    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="offline" containerID="cri-o://c4d75f3d609b7654fb6b5e1c5b1f6af6c12aab44ea5479205f80051ac890eeed" gracePeriod=30
May 29 03:15:48.777713 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:15:48.777668    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="online" containerID="cri-o://be6cac882c6a640588d9e60c5cd7cdc68951fd698175ee554e159a5f5809ee4f" gracePeriod=30
May 29 03:15:48.777713 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:15:48.777692    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="ui" containerID="cri-o://87825e635f99b6112d50390a1f8d6011e0a5500bfc794bd9b616456267515655" gracePeriod=30
May 29 03:15:50.040587 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:15:50.040555    2559 generic.go:358] "Generic (PLEG): container finished" podID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerID="351d08111ac244cb515fd91c01d77d19c39b9d3a763749528a13076229dd8f90" exitCode=0
May 29 03:15:50.040587 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:15:50.040580    2559 generic.go:358] "Generic (PLEG): container finished" podID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerID="532af0022c192883f47649d7b9721d26d25272b71e99be118d862ec73dab9a72" exitCode=0
May 29 03:15:50.041084 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:15:50.040629    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4" event={"ID":"4b928ca2-f0bb-41d2-9e34-1f6f2f787449","Type":"ContainerDied","Data":"351d08111ac244cb515fd91c01d77d19c39b9d3a763749528a13076229dd8f90"}
May 29 03:15:50.041084 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:15:50.040664    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4" event={"ID":"4b928ca2-f0bb-41d2-9e34-1f6f2f787449","Type":"ContainerDied","Data":"532af0022c192883f47649d7b9721d26d25272b71e99be118d862ec73dab9a72"}
May 29 03:15:50.043044 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:15:50.043025    2559 generic.go:358] "Generic (PLEG): container finished" podID="33a9650c-df7c-4082-b06b-ac0c33850488" containerID="87825e635f99b6112d50390a1f8d6011e0a5500bfc794bd9b616456267515655" exitCode=0
May 29 03:15:50.043044 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:15:50.043040    2559 generic.go:358] "Generic (PLEG): container finished" podID="33a9650c-df7c-4082-b06b-ac0c33850488" containerID="be6cac882c6a640588d9e60c5cd7cdc68951fd698175ee554e159a5f5809ee4f" exitCode=0
May 29 03:15:50.043187 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:15:50.043057    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" event={"ID":"33a9650c-df7c-4082-b06b-ac0c33850488","Type":"ContainerDied","Data":"87825e635f99b6112d50390a1f8d6011e0a5500bfc794bd9b616456267515655"}
May 29 03:15:50.043187 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:15:50.043073    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" event={"ID":"33a9650c-df7c-4082-b06b-ac0c33850488","Type":"ContainerDied","Data":"be6cac882c6a640588d9e60c5cd7cdc68951fd698175ee554e159a5f5809ee4f"}
May 29 03:15:53.969312 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:15:53.969266    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused"
May 29 03:15:54.599576 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:15:54.599537    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused"
May 29 03:15:55.639348 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:15:55.639303    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4" podUID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused"
May 29 03:15:55.857400 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:15:55.857361    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4" podUID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused"
May 29 03:16:03.969331 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:03.969278    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused"
May 29 03:16:04.599700 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:04.599664    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused"
May 29 03:16:05.639306 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:05.639265    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4" podUID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused"
May 29 03:16:05.857250 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:05.857212    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4" podUID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused"
May 29 03:16:13.969212 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:13.969160    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused"
May 29 03:16:13.969639 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:13.969294    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:16:14.599503 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:14.599465    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused"
May 29 03:16:14.599668 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:14.599570    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:16:15.638862 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:15.638816    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4" podUID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused"
May 29 03:16:15.639274 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:15.638950    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:16:15.857588 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:15.857551    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4" podUID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused"
May 29 03:16:15.857756 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:15.857655    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:16:19.131939 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.131908    2559 generic.go:358] "Generic (PLEG): container finished" podID="33a9650c-df7c-4082-b06b-ac0c33850488" containerID="c4d75f3d609b7654fb6b5e1c5b1f6af6c12aab44ea5479205f80051ac890eeed" exitCode=137
May 29 03:16:19.131939 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.131935    2559 generic.go:358] "Generic (PLEG): container finished" podID="33a9650c-df7c-4082-b06b-ac0c33850488" containerID="4a845d76fd10290a925d836f8a1298dc1744140028a4c2030ed02c5a896ed4a9" exitCode=137
May 29 03:16:19.132430 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.131992    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" event={"ID":"33a9650c-df7c-4082-b06b-ac0c33850488","Type":"ContainerDied","Data":"c4d75f3d609b7654fb6b5e1c5b1f6af6c12aab44ea5479205f80051ac890eeed"}
May 29 03:16:19.132430 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.132028    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" event={"ID":"33a9650c-df7c-4082-b06b-ac0c33850488","Type":"ContainerDied","Data":"4a845d76fd10290a925d836f8a1298dc1744140028a4c2030ed02c5a896ed4a9"}
May 29 03:16:19.134001 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.133981    2559 generic.go:358] "Generic (PLEG): container finished" podID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerID="c0e21fdc829e525a9e97631e8176d0809b3463c0ec148a05d3532fc542ad52e8" exitCode=137
May 29 03:16:19.134001 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.133993    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4" event={"ID":"4b928ca2-f0bb-41d2-9e34-1f6f2f787449","Type":"ContainerDied","Data":"c0e21fdc829e525a9e97631e8176d0809b3463c0ec148a05d3532fc542ad52e8"}
May 29 03:16:19.280236 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.280215    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:16:19.385574 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.385502    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-feast-data\") pod \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\" (UID: \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\") "
May 29 03:16:19.385574 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.385544    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-online-tls\") pod \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\" (UID: \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\") "
May 29 03:16:19.385772 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.385602    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84jxs\" (UniqueName: \"kubernetes.io/projected/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-kube-api-access-84jxs\") pod \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\" (UID: \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\") "
May 29 03:16:19.385772 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.385622    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-registry-tls\") pod \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\" (UID: \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\") "
May 29 03:16:19.385772 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.385655    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-offline-tls\") pod \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\" (UID: \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\") "
May 29 03:16:19.385772 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.385685    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-ui-tls\") pod \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\" (UID: \"4b928ca2-f0bb-41d2-9e34-1f6f2f787449\") "
May 29 03:16:19.386091 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.386051    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4b928ca2-f0bb-41d2-9e34-1f6f2f787449" (UID: "4b928ca2-f0bb-41d2-9e34-1f6f2f787449"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 03:16:19.386229 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.386135    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-feast-data" (OuterVolumeSpecName: "feast-data") pod "4b928ca2-f0bb-41d2-9e34-1f6f2f787449" (UID: "4b928ca2-f0bb-41d2-9e34-1f6f2f787449"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 03:16:19.388615 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.388584    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-kube-api-access-84jxs" (OuterVolumeSpecName: "kube-api-access-84jxs") pod "4b928ca2-f0bb-41d2-9e34-1f6f2f787449" (UID: "4b928ca2-f0bb-41d2-9e34-1f6f2f787449"). InnerVolumeSpecName "kube-api-access-84jxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 03:16:19.388717 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.388650    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-online-tls" (OuterVolumeSpecName: "online-tls") pod "4b928ca2-f0bb-41d2-9e34-1f6f2f787449" (UID: "4b928ca2-f0bb-41d2-9e34-1f6f2f787449"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 03:16:19.388717 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.388667    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "4b928ca2-f0bb-41d2-9e34-1f6f2f787449" (UID: "4b928ca2-f0bb-41d2-9e34-1f6f2f787449"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 03:16:19.388717 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.388700    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "4b928ca2-f0bb-41d2-9e34-1f6f2f787449" (UID: "4b928ca2-f0bb-41d2-9e34-1f6f2f787449"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 03:16:19.415852 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.415832    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:16:19.486918 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.486892    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/33a9650c-df7c-4082-b06b-ac0c33850488-online-tls\") pod \"33a9650c-df7c-4082-b06b-ac0c33850488\" (UID: \"33a9650c-df7c-4082-b06b-ac0c33850488\") "
May 29 03:16:19.487047 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.486940    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/33a9650c-df7c-4082-b06b-ac0c33850488-offline-tls\") pod \"33a9650c-df7c-4082-b06b-ac0c33850488\" (UID: \"33a9650c-df7c-4082-b06b-ac0c33850488\") "
May 29 03:16:19.487047 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.487000    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/33a9650c-df7c-4082-b06b-ac0c33850488-feast-data\") pod \"33a9650c-df7c-4082-b06b-ac0c33850488\" (UID: \"33a9650c-df7c-4082-b06b-ac0c33850488\") "
May 29 03:16:19.487047 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.487025    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/33a9650c-df7c-4082-b06b-ac0c33850488-registry-tls\") pod \"33a9650c-df7c-4082-b06b-ac0c33850488\" (UID: \"33a9650c-df7c-4082-b06b-ac0c33850488\") "
May 29 03:16:19.487162 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.487081    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/33a9650c-df7c-4082-b06b-ac0c33850488-ui-tls\") pod \"33a9650c-df7c-4082-b06b-ac0c33850488\" (UID: \"33a9650c-df7c-4082-b06b-ac0c33850488\") "
May 29 03:16:19.487162 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.487114    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqxks\" (UniqueName: \"kubernetes.io/projected/33a9650c-df7c-4082-b06b-ac0c33850488-kube-api-access-pqxks\") pod \"33a9650c-df7c-4082-b06b-ac0c33850488\" (UID: \"33a9650c-df7c-4082-b06b-ac0c33850488\") "
May 29 03:16:19.487385 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.487363    2559 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-ui-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:16:19.487385 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.487386    2559 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-feast-data\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:16:19.487564 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.487400    2559 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-online-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:16:19.487564 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.487412    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84jxs\" (UniqueName: \"kubernetes.io/projected/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-kube-api-access-84jxs\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:16:19.487564 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.487426    2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-registry-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:16:19.487564 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.487440    2559 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4b928ca2-f0bb-41d2-9e34-1f6f2f787449-offline-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:16:19.487564 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.487539    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33a9650c-df7c-4082-b06b-ac0c33850488-feast-data" (OuterVolumeSpecName: "feast-data") pod "33a9650c-df7c-4082-b06b-ac0c33850488" (UID: "33a9650c-df7c-4082-b06b-ac0c33850488"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 03:16:19.489073 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.489051    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33a9650c-df7c-4082-b06b-ac0c33850488-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "33a9650c-df7c-4082-b06b-ac0c33850488" (UID: "33a9650c-df7c-4082-b06b-ac0c33850488"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 03:16:19.489521 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.489499    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33a9650c-df7c-4082-b06b-ac0c33850488-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "33a9650c-df7c-4082-b06b-ac0c33850488" (UID: "33a9650c-df7c-4082-b06b-ac0c33850488"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 03:16:19.489589 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.489552    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33a9650c-df7c-4082-b06b-ac0c33850488-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "33a9650c-df7c-4082-b06b-ac0c33850488" (UID: "33a9650c-df7c-4082-b06b-ac0c33850488"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 03:16:19.489589 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.489569    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33a9650c-df7c-4082-b06b-ac0c33850488-kube-api-access-pqxks" (OuterVolumeSpecName: "kube-api-access-pqxks") pod "33a9650c-df7c-4082-b06b-ac0c33850488" (UID: "33a9650c-df7c-4082-b06b-ac0c33850488"). InnerVolumeSpecName "kube-api-access-pqxks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 03:16:19.489655 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.489569    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33a9650c-df7c-4082-b06b-ac0c33850488-online-tls" (OuterVolumeSpecName: "online-tls") pod "33a9650c-df7c-4082-b06b-ac0c33850488" (UID: "33a9650c-df7c-4082-b06b-ac0c33850488"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 03:16:19.587984 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.587948    2559 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/33a9650c-df7c-4082-b06b-ac0c33850488-online-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:16:19.588080 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.587985    2559 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/33a9650c-df7c-4082-b06b-ac0c33850488-offline-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:16:19.588080 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.587999    2559 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/33a9650c-df7c-4082-b06b-ac0c33850488-feast-data\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:16:19.588080 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.588012    2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/33a9650c-df7c-4082-b06b-ac0c33850488-registry-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:16:19.588080 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.588023    2559 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/33a9650c-df7c-4082-b06b-ac0c33850488-ui-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:16:19.588080 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:19.588031    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqxks\" (UniqueName: \"kubernetes.io/projected/33a9650c-df7c-4082-b06b-ac0c33850488-kube-api-access-pqxks\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:16:20.139233 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:20.139197    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v" event={"ID":"33a9650c-df7c-4082-b06b-ac0c33850488","Type":"ContainerDied","Data":"20140be46d4219c2f97133f622b21392e9df930fdc0b50e5343029783e26737c"}
May 29 03:16:20.139681 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:20.139247    2559 scope.go:117] "RemoveContainer" containerID="87825e635f99b6112d50390a1f8d6011e0a5500bfc794bd9b616456267515655"
May 29 03:16:20.139681 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:20.139209    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"
May 29 03:16:20.141659 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:20.141623    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4" event={"ID":"4b928ca2-f0bb-41d2-9e34-1f6f2f787449","Type":"ContainerDied","Data":"1d6d03252e1129de41bccb52b1281a98b00e4ae762dd7e238c1242fc0f0e3797"}
May 29 03:16:20.141765 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:20.141709    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"
May 29 03:16:20.148161 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:20.148143    2559 scope.go:117] "RemoveContainer" containerID="c4d75f3d609b7654fb6b5e1c5b1f6af6c12aab44ea5479205f80051ac890eeed"
May 29 03:16:20.155078 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:20.155049    2559 scope.go:117] "RemoveContainer" containerID="be6cac882c6a640588d9e60c5cd7cdc68951fd698175ee554e159a5f5809ee4f"
May 29 03:16:20.159747 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:20.159728    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"]
May 29 03:16:20.162866 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:20.162850    2559 scope.go:117] "RemoveContainer" containerID="4a845d76fd10290a925d836f8a1298dc1744140028a4c2030ed02c5a896ed4a9"
May 29 03:16:20.163562 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:20.163534    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-zk95v"]
May 29 03:16:20.169501 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:20.169486    2559 scope.go:117] "RemoveContainer" containerID="accc65cfa4b91b525de66c051300a7c7c6d4db5552798f29ed97d4f5153fbb51"
May 29 03:16:20.172997 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:20.172962    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"]
May 29 03:16:20.176484 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:20.176464    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-nn2z4"]
May 29 03:16:20.176804 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:20.176790    2559 scope.go:117] "RemoveContainer" containerID="5f9129dfbe9ed58999a25da4d45be7ee9adb33898553f0c840541d3f9423e8d2"
May 29 03:16:20.183384 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:20.183365    2559 scope.go:117] "RemoveContainer" containerID="351d08111ac244cb515fd91c01d77d19c39b9d3a763749528a13076229dd8f90"
May 29 03:16:20.190105 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:20.190091    2559 scope.go:117] "RemoveContainer" containerID="c0e21fdc829e525a9e97631e8176d0809b3463c0ec148a05d3532fc542ad52e8"
May 29 03:16:20.196353 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:20.196338    2559 scope.go:117] "RemoveContainer" containerID="532af0022c192883f47649d7b9721d26d25272b71e99be118d862ec73dab9a72"
May 29 03:16:20.202898 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:20.202883    2559 scope.go:117] "RemoveContainer" containerID="420554f1e6118c2dac9459193a508f34a7e08ce66f8064c9f65fd01cfb70f739"
May 29 03:16:20.209134 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:20.209121    2559 scope.go:117] "RemoveContainer" containerID="bb2386ca2c05a00997078dd79f289a812d1079b97c88a5873df4bc442931b3d8"
May 29 03:16:21.848284 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:21.848250    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" path="/var/lib/kubelet/pods/33a9650c-df7c-4082-b06b-ac0c33850488/volumes"
May 29 03:16:21.848846 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:16:21.848832    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" path="/var/lib/kubelet/pods/4b928ca2-f0bb-41d2-9e34-1f6f2f787449/volumes"
May 29 03:17:00.272796 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.272761    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"]
May 29 03:17:00.273228 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273208    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerName="ui"
May 29 03:17:00.273228 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273227    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerName="ui"
May 29 03:17:00.273303 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273242    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerName="feast-apply"
May 29 03:17:00.273303 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273249    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerName="feast-apply"
May 29 03:17:00.273303 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273263    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="registry"
May 29 03:17:00.273303 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273271    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="registry"
May 29 03:17:00.273303 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273285    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="feast-init"
May 29 03:17:00.273303 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273293    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="feast-init"
May 29 03:17:00.273529 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273303    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="offline"
May 29 03:17:00.273529 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273312    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="offline"
May 29 03:17:00.273529 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273321    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerName="offline"
May 29 03:17:00.273529 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273329    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerName="offline"
May 29 03:17:00.273529 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273339    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="feast-apply"
May 29 03:17:00.273529 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273347    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="feast-apply"
May 29 03:17:00.273529 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273357    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerName="feast-init"
May 29 03:17:00.273529 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273366    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerName="feast-init"
May 29 03:17:00.273529 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273375    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="online"
May 29 03:17:00.273529 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273383    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="online"
May 29 03:17:00.273529 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273392    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="ui"
May 29 03:17:00.273529 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273400    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="ui"
May 29 03:17:00.273529 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273415    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerName="online"
May 29 03:17:00.273529 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273423    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerName="online"
May 29 03:17:00.273529 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273500    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerName="offline"
May 29 03:17:00.273529 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273509    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerName="ui"
May 29 03:17:00.273529 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273518    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="online"
May 29 03:17:00.273529 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273530    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="offline"
May 29 03:17:00.274049 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273540    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="4b928ca2-f0bb-41d2-9e34-1f6f2f787449" containerName="online"
May 29 03:17:00.274049 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273550    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="registry"
May 29 03:17:00.274049 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.273559    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="33a9650c-df7c-4082-b06b-ac0c33850488" containerName="ui"
May 29 03:17:00.277061 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.277045    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:00.279242 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.279223    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 29 03:17:00.279458 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.279437    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 29 03:17:00.279536 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.279450    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 29 03:17:00.279589 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.279534    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 29 03:17:00.279817 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.279799    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-9tt5s\""
May 29 03:17:00.279917 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.279833    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 29 03:17:00.279917 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.279831    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 29 03:17:00.286767 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.286745    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"]
May 29 03:17:00.387835 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.387805    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/eb5251c7-1d51-466e-a5ff-817e3c815020-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-scbk8\" (UID: \"eb5251c7-1d51-466e-a5ff-817e3c815020\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:00.388000 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.387842    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/eb5251c7-1d51-466e-a5ff-817e3c815020-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-scbk8\" (UID: \"eb5251c7-1d51-466e-a5ff-817e3c815020\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:00.388000 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.387863    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v469z\" (UniqueName: \"kubernetes.io/projected/eb5251c7-1d51-466e-a5ff-817e3c815020-kube-api-access-v469z\") pod \"feast-simple-feast-setup-664fcfcc58-scbk8\" (UID: \"eb5251c7-1d51-466e-a5ff-817e3c815020\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:00.388000 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.387883    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/eb5251c7-1d51-466e-a5ff-817e3c815020-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-scbk8\" (UID: \"eb5251c7-1d51-466e-a5ff-817e3c815020\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:00.388000 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.387907    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/eb5251c7-1d51-466e-a5ff-817e3c815020-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-scbk8\" (UID: \"eb5251c7-1d51-466e-a5ff-817e3c815020\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:00.388000 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.387949    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/eb5251c7-1d51-466e-a5ff-817e3c815020-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-scbk8\" (UID: \"eb5251c7-1d51-466e-a5ff-817e3c815020\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:00.489094 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.489066    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/eb5251c7-1d51-466e-a5ff-817e3c815020-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-scbk8\" (UID: \"eb5251c7-1d51-466e-a5ff-817e3c815020\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:00.489252 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.489100    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/eb5251c7-1d51-466e-a5ff-817e3c815020-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-scbk8\" (UID: \"eb5251c7-1d51-466e-a5ff-817e3c815020\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:00.489252 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.489121    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/eb5251c7-1d51-466e-a5ff-817e3c815020-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-scbk8\" (UID: \"eb5251c7-1d51-466e-a5ff-817e3c815020\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:00.489371 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.489259    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/eb5251c7-1d51-466e-a5ff-817e3c815020-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-scbk8\" (UID: \"eb5251c7-1d51-466e-a5ff-817e3c815020\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:00.489371 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.489312    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/eb5251c7-1d51-466e-a5ff-817e3c815020-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-scbk8\" (UID: \"eb5251c7-1d51-466e-a5ff-817e3c815020\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:00.489371 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.489343    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v469z\" (UniqueName: \"kubernetes.io/projected/eb5251c7-1d51-466e-a5ff-817e3c815020-kube-api-access-v469z\") pod \"feast-simple-feast-setup-664fcfcc58-scbk8\" (UID: \"eb5251c7-1d51-466e-a5ff-817e3c815020\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:00.489644 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.489623    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/eb5251c7-1d51-466e-a5ff-817e3c815020-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-scbk8\" (UID: \"eb5251c7-1d51-466e-a5ff-817e3c815020\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:00.491574 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.491545    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/eb5251c7-1d51-466e-a5ff-817e3c815020-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-scbk8\" (UID: \"eb5251c7-1d51-466e-a5ff-817e3c815020\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:00.491662 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.491619    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/eb5251c7-1d51-466e-a5ff-817e3c815020-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-scbk8\" (UID: \"eb5251c7-1d51-466e-a5ff-817e3c815020\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:00.491873 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.491858    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/eb5251c7-1d51-466e-a5ff-817e3c815020-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-scbk8\" (UID: \"eb5251c7-1d51-466e-a5ff-817e3c815020\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:00.491910 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.491896    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/eb5251c7-1d51-466e-a5ff-817e3c815020-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-scbk8\" (UID: \"eb5251c7-1d51-466e-a5ff-817e3c815020\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:00.496336 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.496315    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v469z\" (UniqueName: \"kubernetes.io/projected/eb5251c7-1d51-466e-a5ff-817e3c815020-kube-api-access-v469z\") pod \"feast-simple-feast-setup-664fcfcc58-scbk8\" (UID: \"eb5251c7-1d51-466e-a5ff-817e3c815020\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:00.595885 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.595834    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:00.718421 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.718346    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"]
May 29 03:17:00.720835 ip-10-0-137-122 kubenswrapper[2559]: W0529 03:17:00.720805    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb5251c7_1d51_466e_a5ff_817e3c815020.slice/crio-2fd54fde32ff967d5abb8e5635ab12060777ac146743bae1a3597dbf77463878 WatchSource:0}: Error finding container 2fd54fde32ff967d5abb8e5635ab12060777ac146743bae1a3597dbf77463878: Status 404 returned error can't find the container with id 2fd54fde32ff967d5abb8e5635ab12060777ac146743bae1a3597dbf77463878
May 29 03:17:00.722769 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:00.722749    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 29 03:17:01.258119 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:01.258080    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" event={"ID":"eb5251c7-1d51-466e-a5ff-817e3c815020","Type":"ContainerStarted","Data":"b784fff78783ca646e5cab9d8e4d3f81c1d2d681d89de1278e1e9900bc3bd304"}
May 29 03:17:01.258119 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:01.258122    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" event={"ID":"eb5251c7-1d51-466e-a5ff-817e3c815020","Type":"ContainerStarted","Data":"2fd54fde32ff967d5abb8e5635ab12060777ac146743bae1a3597dbf77463878"}
May 29 03:17:05.271610 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:05.271578    2559 generic.go:358] "Generic (PLEG): container finished" podID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerID="b784fff78783ca646e5cab9d8e4d3f81c1d2d681d89de1278e1e9900bc3bd304" exitCode=0
May 29 03:17:05.271936 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:05.271653    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" event={"ID":"eb5251c7-1d51-466e-a5ff-817e3c815020","Type":"ContainerDied","Data":"b784fff78783ca646e5cab9d8e4d3f81c1d2d681d89de1278e1e9900bc3bd304"}
May 29 03:17:06.276181 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:06.276149    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" event={"ID":"eb5251c7-1d51-466e-a5ff-817e3c815020","Type":"ContainerStarted","Data":"7112d35a50859936ce3da87c8bd6a7987b36ce5d8f15f8a6e750faa146070d56"}
May 29 03:17:11.291840 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:11.291807    2559 generic.go:358] "Generic (PLEG): container finished" podID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerID="7112d35a50859936ce3da87c8bd6a7987b36ce5d8f15f8a6e750faa146070d56" exitCode=0
May 29 03:17:11.292244 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:11.291872    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" event={"ID":"eb5251c7-1d51-466e-a5ff-817e3c815020","Type":"ContainerDied","Data":"7112d35a50859936ce3da87c8bd6a7987b36ce5d8f15f8a6e750faa146070d56"}
May 29 03:17:12.299078 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:12.299038    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" event={"ID":"eb5251c7-1d51-466e-a5ff-817e3c815020","Type":"ContainerStarted","Data":"01da7a2a81d9ef3effc1fdaa53fbd7bdfa377b28d0aa2e9bd1ada1cab6589604"}
May 29 03:17:12.299078 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:12.299085    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" event={"ID":"eb5251c7-1d51-466e-a5ff-817e3c815020","Type":"ContainerStarted","Data":"79cae18d9a89c55b8510365b241a301018378dce7b53ffea4106c9b8d40c33c0"}
May 29 03:17:12.299606 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:12.299100    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" event={"ID":"eb5251c7-1d51-466e-a5ff-817e3c815020","Type":"ContainerStarted","Data":"f9989e00b85a7571bbb554fc21e1a7f78a2bb0dad5c67481c3942fdb7d21f188"}
May 29 03:17:12.299606 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:12.299115    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" event={"ID":"eb5251c7-1d51-466e-a5ff-817e3c815020","Type":"ContainerStarted","Data":"d8ae2e45ea0b56e69522c8286e36bd4d2a54bf832b98691c530c67fe26e10331"}
May 29 03:17:12.321183 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:12.321082    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" podStartSLOduration=12.321063896 podStartE2EDuration="12.321063896s" podCreationTimestamp="2026-05-29 03:17:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 03:17:12.320134376 +0000 UTC m=+1483.075449455" watchObservedRunningTime="2026-05-29 03:17:12.321063896 +0000 UTC m=+1483.076378962"
May 29 03:17:12.597107 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:12.596977    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:12.597107 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:12.597029    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:12.597107 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:12.597043    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:12.597107 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:12.597055    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:12.599198 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:12.599146    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.26:6571: connect: connection refused"
May 29 03:17:12.599198 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:12.599181    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused"
May 29 03:17:12.599394 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:12.599158    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.26:8816: connect: connection refused"
May 29 03:17:12.599558 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:12.599529    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused"
May 29 03:17:15.596539 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:15.596492    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.26:6571: connect: connection refused"
May 29 03:17:15.597185 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:15.596514    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused"
May 29 03:17:15.597332 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:15.596572    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.26:8816: connect: connection refused"
May 29 03:17:15.597332 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:15.596578    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused"
May 29 03:17:18.597206 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:18.597168    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:18.597619 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:18.597308    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:18.597619 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:18.597535    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:18.597793 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:18.597775    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:18.597856 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:18.597801    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:18.597856 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:18.597814    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:18.598114 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:18.598096    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:18.598166 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:18.598158    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:18.598421 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:18.598405    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:18.601983 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:18.601952    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:19.323027 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:19.322999    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:19.325814 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:19.325796    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:17:29.854651 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:29.854626    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psj9n_d9a08de9-bea6-4561-8eae-62421b7584a9/ovn-acl-logging/0.log"
May 29 03:17:29.855519 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:17:29.855500    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psj9n_d9a08de9-bea6-4561-8eae-62421b7584a9/ovn-acl-logging/0.log"
May 29 03:18:22.018925 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:22.018892    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"]
May 29 03:18:22.019453 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:22.019347    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="registry" containerID="cri-o://d8ae2e45ea0b56e69522c8286e36bd4d2a54bf832b98691c530c67fe26e10331" gracePeriod=30
May 29 03:18:22.019527 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:22.019425    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="online" containerID="cri-o://f9989e00b85a7571bbb554fc21e1a7f78a2bb0dad5c67481c3942fdb7d21f188" gracePeriod=30
May 29 03:18:22.019527 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:22.019422    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="offline" containerID="cri-o://79cae18d9a89c55b8510365b241a301018378dce7b53ffea4106c9b8d40c33c0" gracePeriod=30
May 29 03:18:22.019527 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:22.019453    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="ui" containerID="cri-o://01da7a2a81d9ef3effc1fdaa53fbd7bdfa377b28d0aa2e9bd1ada1cab6589604" gracePeriod=30
May 29 03:18:22.306029 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:22.305915    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"]
May 29 03:18:22.310937 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:22.310910    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:22.312909 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:22.312889    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-gmdrh\""
May 29 03:18:22.324345 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:22.324324    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"]
May 29 03:18:22.432477 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:22.432440    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-858km\" (UID: \"00547dba-08f8-441a-8aff-e82990542301\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:22.432697 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:22.432487    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/00547dba-08f8-441a-8aff-e82990542301-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-858km\" (UID: \"00547dba-08f8-441a-8aff-e82990542301\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:22.432697 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:22.432561    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-858km\" (UID: \"00547dba-08f8-441a-8aff-e82990542301\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:22.432697 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:22.432682    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-858km\" (UID: \"00547dba-08f8-441a-8aff-e82990542301\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:22.432907 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:22.432727    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c9qt\" (UniqueName: \"kubernetes.io/projected/00547dba-08f8-441a-8aff-e82990542301-kube-api-access-4c9qt\") pod \"feast-simple-feast-setup-664fcfcc58-858km\" (UID: \"00547dba-08f8-441a-8aff-e82990542301\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:22.432907 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:22.432787    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-858km\" (UID: \"00547dba-08f8-441a-8aff-e82990542301\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:22.533949 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:22.533918    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-858km\" (UID: \"00547dba-08f8-441a-8aff-e82990542301\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:22.533949 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:22.533954    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/00547dba-08f8-441a-8aff-e82990542301-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-858km\" (UID: \"00547dba-08f8-441a-8aff-e82990542301\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:22.534182 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:22.534008    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-858km\" (UID: \"00547dba-08f8-441a-8aff-e82990542301\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:22.534182 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:22.534064    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-858km\" (UID: \"00547dba-08f8-441a-8aff-e82990542301\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:22.534182 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:22.534093    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4c9qt\" (UniqueName: \"kubernetes.io/projected/00547dba-08f8-441a-8aff-e82990542301-kube-api-access-4c9qt\") pod \"feast-simple-feast-setup-664fcfcc58-858km\" (UID: \"00547dba-08f8-441a-8aff-e82990542301\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:22.534182 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:22.534123    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-858km\" (UID: \"00547dba-08f8-441a-8aff-e82990542301\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:22.534182 ip-10-0-137-122 kubenswrapper[2559]: E0529 03:18:22.534173    2559 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found
May 29 03:18:22.534437 ip-10-0-137-122 kubenswrapper[2559]: E0529 03:18:22.534259    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-registry-tls podName:00547dba-08f8-441a-8aff-e82990542301 nodeName:}" failed. No retries permitted until 2026-05-29 03:18:23.034237037 +0000 UTC m=+1553.789552085 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-registry-tls") pod "feast-simple-feast-setup-664fcfcc58-858km" (UID: "00547dba-08f8-441a-8aff-e82990542301") : secret "feast-simple-feast-setup-registry-tls" not found
May 29 03:18:22.534525 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:22.534441    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/00547dba-08f8-441a-8aff-e82990542301-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-858km\" (UID: \"00547dba-08f8-441a-8aff-e82990542301\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:22.536776 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:22.536755    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-858km\" (UID: \"00547dba-08f8-441a-8aff-e82990542301\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:22.536999 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:22.536957    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-858km\" (UID: \"00547dba-08f8-441a-8aff-e82990542301\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:22.537112 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:22.537092    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-858km\" (UID: \"00547dba-08f8-441a-8aff-e82990542301\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:22.541527 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:22.541497    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c9qt\" (UniqueName: \"kubernetes.io/projected/00547dba-08f8-441a-8aff-e82990542301-kube-api-access-4c9qt\") pod \"feast-simple-feast-setup-664fcfcc58-858km\" (UID: \"00547dba-08f8-441a-8aff-e82990542301\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:23.039861 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:23.039774    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-858km\" (UID: \"00547dba-08f8-441a-8aff-e82990542301\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:23.042104 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:23.042077    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-858km\" (UID: \"00547dba-08f8-441a-8aff-e82990542301\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:23.225213 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:23.225186    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:23.352404 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:23.352375    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"]
May 29 03:18:23.354886 ip-10-0-137-122 kubenswrapper[2559]: W0529 03:18:23.354859    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00547dba_08f8_441a_8aff_e82990542301.slice/crio-48cf2916aa3227cb99abbe6cec1f339607f808fddae4598fe1da85227ba77942 WatchSource:0}: Error finding container 48cf2916aa3227cb99abbe6cec1f339607f808fddae4598fe1da85227ba77942: Status 404 returned error can't find the container with id 48cf2916aa3227cb99abbe6cec1f339607f808fddae4598fe1da85227ba77942
May 29 03:18:23.514353 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:23.514311    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" event={"ID":"00547dba-08f8-441a-8aff-e82990542301","Type":"ContainerStarted","Data":"92fd35e0cb3fc7d0c9155a7e56b942a4b08416e8ff560089961b2f16b67f383b"}
May 29 03:18:23.514549 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:23.514365    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" event={"ID":"00547dba-08f8-441a-8aff-e82990542301","Type":"ContainerStarted","Data":"48cf2916aa3227cb99abbe6cec1f339607f808fddae4598fe1da85227ba77942"}
May 29 03:18:23.516899 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:23.516865    2559 generic.go:358] "Generic (PLEG): container finished" podID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerID="01da7a2a81d9ef3effc1fdaa53fbd7bdfa377b28d0aa2e9bd1ada1cab6589604" exitCode=0
May 29 03:18:23.516899 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:23.516886    2559 generic.go:358] "Generic (PLEG): container finished" podID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerID="f9989e00b85a7571bbb554fc21e1a7f78a2bb0dad5c67481c3942fdb7d21f188" exitCode=0
May 29 03:18:23.517057 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:23.516908    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" event={"ID":"eb5251c7-1d51-466e-a5ff-817e3c815020","Type":"ContainerDied","Data":"01da7a2a81d9ef3effc1fdaa53fbd7bdfa377b28d0aa2e9bd1ada1cab6589604"}
May 29 03:18:23.517057 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:23.516930    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" event={"ID":"eb5251c7-1d51-466e-a5ff-817e3c815020","Type":"ContainerDied","Data":"f9989e00b85a7571bbb554fc21e1a7f78a2bb0dad5c67481c3942fdb7d21f188"}
May 29 03:18:28.533923 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:28.533892    2559 generic.go:358] "Generic (PLEG): container finished" podID="00547dba-08f8-441a-8aff-e82990542301" containerID="92fd35e0cb3fc7d0c9155a7e56b942a4b08416e8ff560089961b2f16b67f383b" exitCode=0
May 29 03:18:28.534285 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:28.533987    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" event={"ID":"00547dba-08f8-441a-8aff-e82990542301","Type":"ContainerDied","Data":"92fd35e0cb3fc7d0c9155a7e56b942a4b08416e8ff560089961b2f16b67f383b"}
May 29 03:18:28.598609 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:28.598567    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused"
May 29 03:18:29.323434 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:29.323395    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused"
May 29 03:18:29.538342 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:29.538311    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" event={"ID":"00547dba-08f8-441a-8aff-e82990542301","Type":"ContainerStarted","Data":"a80e30027f8bccd20ec2a5396fa9b37a0ede2ce0bc6ae5951ce9e7628073e246"}
May 29 03:18:34.554568 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:34.554535    2559 generic.go:358] "Generic (PLEG): container finished" podID="00547dba-08f8-441a-8aff-e82990542301" containerID="a80e30027f8bccd20ec2a5396fa9b37a0ede2ce0bc6ae5951ce9e7628073e246" exitCode=0
May 29 03:18:34.554992 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:34.554607    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" event={"ID":"00547dba-08f8-441a-8aff-e82990542301","Type":"ContainerDied","Data":"a80e30027f8bccd20ec2a5396fa9b37a0ede2ce0bc6ae5951ce9e7628073e246"}
May 29 03:18:35.562212 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:35.562165    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" event={"ID":"00547dba-08f8-441a-8aff-e82990542301","Type":"ContainerStarted","Data":"be85097d6dba776169f2fcd12df7f641ca8ade9807e4d8568240b281462cdc5e"}
May 29 03:18:35.562688 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:35.562227    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" event={"ID":"00547dba-08f8-441a-8aff-e82990542301","Type":"ContainerStarted","Data":"9a7bf818389c22039c5ead80e9f89bf50a4000f6a16d2139118afa1986ae17bf"}
May 29 03:18:35.562688 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:35.562243    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" event={"ID":"00547dba-08f8-441a-8aff-e82990542301","Type":"ContainerStarted","Data":"6065da4dba7c9d8d35c8db52205dea642fe89e835d4f9a0c50378d8d37b0a134"}
May 29 03:18:35.562688 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:35.562256    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" event={"ID":"00547dba-08f8-441a-8aff-e82990542301","Type":"ContainerStarted","Data":"dd2776e69f85668097bb4d369288416b42a5cd681278c7a40d51ada31e800f0d"}
May 29 03:18:35.587655 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:35.587590    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" podStartSLOduration=13.587570131 podStartE2EDuration="13.587570131s" podCreationTimestamp="2026-05-29 03:18:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 03:18:35.584999068 +0000 UTC m=+1566.340314133" watchObservedRunningTime="2026-05-29 03:18:35.587570131 +0000 UTC m=+1566.342885196"
May 29 03:18:38.225878 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:38.225837    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:38.226361 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:38.225894    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:38.226361 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:38.225912    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:38.226361 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:38.225930    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:38.227741 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:38.227703    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.27:8816: connect: connection refused"
May 29 03:18:38.227741 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:38.227718    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused"
May 29 03:18:38.227741 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:38.227721    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.27:6571: connect: connection refused"
May 29 03:18:38.228003 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:38.227827    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused"
May 29 03:18:38.599016 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:38.598929    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused"
May 29 03:18:39.323638 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:39.323590    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused"
May 29 03:18:41.226468 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:41.226438    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:41.226935 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:41.226663    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:41.227074 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:41.227046    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:41.227145 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:41.227085    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:41.227224 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:41.227208    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:41.227372 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:41.227249    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:41.227372 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:41.227276    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:41.231785 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:41.231761    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:41.583833 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:41.583745    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:41.583833 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:41.583787    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:41.584162 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:41.584141    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:41.586439 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:41.586420    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:18:48.598883 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:48.598831    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused"
May 29 03:18:48.599335 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:48.599019    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:18:49.323652 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:49.323612    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused"
May 29 03:18:49.323835 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:49.323736    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:18:52.619599 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:52.619569    2559 generic.go:358] "Generic (PLEG): container finished" podID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerID="79cae18d9a89c55b8510365b241a301018378dce7b53ffea4106c9b8d40c33c0" exitCode=137
May 29 03:18:52.619599 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:52.619593    2559 generic.go:358] "Generic (PLEG): container finished" podID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerID="d8ae2e45ea0b56e69522c8286e36bd4d2a54bf832b98691c530c67fe26e10331" exitCode=137
May 29 03:18:52.619950 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:52.619621    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" event={"ID":"eb5251c7-1d51-466e-a5ff-817e3c815020","Type":"ContainerDied","Data":"79cae18d9a89c55b8510365b241a301018378dce7b53ffea4106c9b8d40c33c0"}
May 29 03:18:52.619950 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:52.619653    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" event={"ID":"eb5251c7-1d51-466e-a5ff-817e3c815020","Type":"ContainerDied","Data":"d8ae2e45ea0b56e69522c8286e36bd4d2a54bf832b98691c530c67fe26e10331"}
May 29 03:18:52.665540 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:52.665521    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:18:52.701833 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:52.701814    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/eb5251c7-1d51-466e-a5ff-817e3c815020-registry-tls\") pod \"eb5251c7-1d51-466e-a5ff-817e3c815020\" (UID: \"eb5251c7-1d51-466e-a5ff-817e3c815020\") "
May 29 03:18:52.701921 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:52.701870    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/eb5251c7-1d51-466e-a5ff-817e3c815020-ui-tls\") pod \"eb5251c7-1d51-466e-a5ff-817e3c815020\" (UID: \"eb5251c7-1d51-466e-a5ff-817e3c815020\") "
May 29 03:18:52.702021 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:52.702000    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/eb5251c7-1d51-466e-a5ff-817e3c815020-offline-tls\") pod \"eb5251c7-1d51-466e-a5ff-817e3c815020\" (UID: \"eb5251c7-1d51-466e-a5ff-817e3c815020\") "
May 29 03:18:52.702078 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:52.702045    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v469z\" (UniqueName: \"kubernetes.io/projected/eb5251c7-1d51-466e-a5ff-817e3c815020-kube-api-access-v469z\") pod \"eb5251c7-1d51-466e-a5ff-817e3c815020\" (UID: \"eb5251c7-1d51-466e-a5ff-817e3c815020\") "
May 29 03:18:52.702134 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:52.702107    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/eb5251c7-1d51-466e-a5ff-817e3c815020-feast-data\") pod \"eb5251c7-1d51-466e-a5ff-817e3c815020\" (UID: \"eb5251c7-1d51-466e-a5ff-817e3c815020\") "
May 29 03:18:52.702190 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:52.702149    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/eb5251c7-1d51-466e-a5ff-817e3c815020-online-tls\") pod \"eb5251c7-1d51-466e-a5ff-817e3c815020\" (UID: \"eb5251c7-1d51-466e-a5ff-817e3c815020\") "
May 29 03:18:52.702704 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:52.702675    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb5251c7-1d51-466e-a5ff-817e3c815020-feast-data" (OuterVolumeSpecName: "feast-data") pod "eb5251c7-1d51-466e-a5ff-817e3c815020" (UID: "eb5251c7-1d51-466e-a5ff-817e3c815020"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 03:18:52.704182 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:52.704147    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb5251c7-1d51-466e-a5ff-817e3c815020-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "eb5251c7-1d51-466e-a5ff-817e3c815020" (UID: "eb5251c7-1d51-466e-a5ff-817e3c815020"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 03:18:52.704182 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:52.704158    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb5251c7-1d51-466e-a5ff-817e3c815020-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "eb5251c7-1d51-466e-a5ff-817e3c815020" (UID: "eb5251c7-1d51-466e-a5ff-817e3c815020"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 03:18:52.704338 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:52.704271    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb5251c7-1d51-466e-a5ff-817e3c815020-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "eb5251c7-1d51-466e-a5ff-817e3c815020" (UID: "eb5251c7-1d51-466e-a5ff-817e3c815020"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 03:18:52.704400 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:52.704377    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb5251c7-1d51-466e-a5ff-817e3c815020-kube-api-access-v469z" (OuterVolumeSpecName: "kube-api-access-v469z") pod "eb5251c7-1d51-466e-a5ff-817e3c815020" (UID: "eb5251c7-1d51-466e-a5ff-817e3c815020"). InnerVolumeSpecName "kube-api-access-v469z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 03:18:52.704672 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:52.704653    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb5251c7-1d51-466e-a5ff-817e3c815020-online-tls" (OuterVolumeSpecName: "online-tls") pod "eb5251c7-1d51-466e-a5ff-817e3c815020" (UID: "eb5251c7-1d51-466e-a5ff-817e3c815020"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 03:18:52.802874 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:52.802805    2559 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/eb5251c7-1d51-466e-a5ff-817e3c815020-ui-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:18:52.802874 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:52.802830    2559 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/eb5251c7-1d51-466e-a5ff-817e3c815020-offline-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:18:52.802874 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:52.802844    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v469z\" (UniqueName: \"kubernetes.io/projected/eb5251c7-1d51-466e-a5ff-817e3c815020-kube-api-access-v469z\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:18:52.802874 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:52.802857    2559 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/eb5251c7-1d51-466e-a5ff-817e3c815020-feast-data\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:18:52.802874 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:52.802869    2559 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/eb5251c7-1d51-466e-a5ff-817e3c815020-online-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:18:52.803139 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:52.802881    2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/eb5251c7-1d51-466e-a5ff-817e3c815020-registry-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:18:53.624712 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:53.624684    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8" event={"ID":"eb5251c7-1d51-466e-a5ff-817e3c815020","Type":"ContainerDied","Data":"2fd54fde32ff967d5abb8e5635ab12060777ac146743bae1a3597dbf77463878"}
May 29 03:18:53.624712 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:53.624708    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"
May 29 03:18:53.625297 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:53.624729    2559 scope.go:117] "RemoveContainer" containerID="01da7a2a81d9ef3effc1fdaa53fbd7bdfa377b28d0aa2e9bd1ada1cab6589604"
May 29 03:18:53.635641 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:53.635620    2559 scope.go:117] "RemoveContainer" containerID="79cae18d9a89c55b8510365b241a301018378dce7b53ffea4106c9b8d40c33c0"
May 29 03:18:53.643051 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:53.643035    2559 scope.go:117] "RemoveContainer" containerID="f9989e00b85a7571bbb554fc21e1a7f78a2bb0dad5c67481c3942fdb7d21f188"
May 29 03:18:53.649858 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:53.649840    2559 scope.go:117] "RemoveContainer" containerID="d8ae2e45ea0b56e69522c8286e36bd4d2a54bf832b98691c530c67fe26e10331"
May 29 03:18:53.651347 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:53.651327    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"]
May 29 03:18:53.655285 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:53.655262    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-scbk8"]
May 29 03:18:53.657033 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:53.657018    2559 scope.go:117] "RemoveContainer" containerID="7112d35a50859936ce3da87c8bd6a7987b36ce5d8f15f8a6e750faa146070d56"
May 29 03:18:53.663449 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:53.663433    2559 scope.go:117] "RemoveContainer" containerID="b784fff78783ca646e5cab9d8e4d3f81c1d2d681d89de1278e1e9900bc3bd304"
May 29 03:18:53.849520 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:18:53.849488    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" path="/var/lib/kubelet/pods/eb5251c7-1d51-466e-a5ff-817e3c815020/volumes"
May 29 03:19:43.899502 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.899444    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"]
May 29 03:19:43.900006 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.899784    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="registry"
May 29 03:19:43.900006 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.899797    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="registry"
May 29 03:19:43.900006 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.899809    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="feast-init"
May 29 03:19:43.900006 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.899815    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="feast-init"
May 29 03:19:43.900006 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.899824    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="online"
May 29 03:19:43.900006 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.899829    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="online"
May 29 03:19:43.900006 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.899835    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="ui"
May 29 03:19:43.900006 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.899841    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="ui"
May 29 03:19:43.900006 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.899849    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="offline"
May 29 03:19:43.900006 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.899854    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="offline"
May 29 03:19:43.900006 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.899867    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="feast-apply"
May 29 03:19:43.900006 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.899872    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="feast-apply"
May 29 03:19:43.900006 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.899923    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="online"
May 29 03:19:43.900006 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.899931    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="offline"
May 29 03:19:43.900006 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.899937    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="ui"
May 29 03:19:43.900006 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.899945    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="eb5251c7-1d51-466e-a5ff-817e3c815020" containerName="registry"
May 29 03:19:43.908784 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.908753    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:19:43.911778 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.911335    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-d6b4r\""
May 29 03:19:43.911778 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.911555    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 29 03:19:43.911778 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.911604    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 29 03:19:43.911778 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.911705    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 29 03:19:43.911778 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.911731    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 29 03:19:43.912163 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.912144    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 29 03:19:43.912329 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.912299    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 29 03:19:43.912795 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.912771    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"]
May 29 03:19:43.982304 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.982278    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-wgbf8\" (UID: \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:19:43.982438 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.982305    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-wgbf8\" (UID: \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:19:43.982438 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.982327    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grq9d\" (UniqueName: \"kubernetes.io/projected/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-kube-api-access-grq9d\") pod \"feast-simple-feast-remote-setup-5cff94746b-wgbf8\" (UID: \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:19:43.982438 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.982395    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-wgbf8\" (UID: \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:19:43.982438 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.982427    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-wgbf8\" (UID: \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:19:43.982640 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:43.982476    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-wgbf8\" (UID: \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:19:44.083096 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:44.083070    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-wgbf8\" (UID: \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:19:44.083233 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:44.083107    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-wgbf8\" (UID: \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:19:44.083233 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:44.083124    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-wgbf8\" (UID: \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:19:44.083233 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:44.083149    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-grq9d\" (UniqueName: \"kubernetes.io/projected/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-kube-api-access-grq9d\") pod \"feast-simple-feast-remote-setup-5cff94746b-wgbf8\" (UID: \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:19:44.083233 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:44.083193    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-wgbf8\" (UID: \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:19:44.083233 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:44.083226    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-wgbf8\" (UID: \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:19:44.083491 ip-10-0-137-122 kubenswrapper[2559]: E0529 03:19:44.083334    2559 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found
May 29 03:19:44.083491 ip-10-0-137-122 kubenswrapper[2559]: E0529 03:19:44.083408    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-ui-tls podName:ac216ba2-d2f5-4d54-b527-c9dd88764cdf nodeName:}" failed. No retries permitted until 2026-05-29 03:19:44.583386427 +0000 UTC m=+1635.338701478 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-ui-tls") pod "feast-simple-feast-remote-setup-5cff94746b-wgbf8" (UID: "ac216ba2-d2f5-4d54-b527-c9dd88764cdf") : secret "feast-simple-feast-remote-setup-ui-tls" not found
May 29 03:19:44.083606 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:44.083496    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-wgbf8\" (UID: \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:19:44.083901 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:44.083883    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-wgbf8\" (UID: \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:19:44.086103 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:44.086088    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-wgbf8\" (UID: \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:19:44.086229 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:44.086210    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-wgbf8\" (UID: \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:19:44.091210 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:44.091189    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grq9d\" (UniqueName: \"kubernetes.io/projected/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-kube-api-access-grq9d\") pod \"feast-simple-feast-remote-setup-5cff94746b-wgbf8\" (UID: \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:19:44.586488 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:44.586456    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-wgbf8\" (UID: \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:19:44.588829 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:44.588804    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-wgbf8\" (UID: \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:19:44.819656 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:44.819615    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:19:44.946672 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:44.946644    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"]
May 29 03:19:44.949249 ip-10-0-137-122 kubenswrapper[2559]: W0529 03:19:44.949224    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac216ba2_d2f5_4d54_b527_c9dd88764cdf.slice/crio-275b266563413ed3c46cd15563e7db2c5075ab8c477f1b5cf74915fb4b05e39b WatchSource:0}: Error finding container 275b266563413ed3c46cd15563e7db2c5075ab8c477f1b5cf74915fb4b05e39b: Status 404 returned error can't find the container with id 275b266563413ed3c46cd15563e7db2c5075ab8c477f1b5cf74915fb4b05e39b
May 29 03:19:45.777850 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:45.777817    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" event={"ID":"ac216ba2-d2f5-4d54-b527-c9dd88764cdf","Type":"ContainerStarted","Data":"4325dd5eb8eea13d70d9f0f070c66909ced03396b31acf3c0639b8b9e1e99197"}
May 29 03:19:45.777850 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:45.777854    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" event={"ID":"ac216ba2-d2f5-4d54-b527-c9dd88764cdf","Type":"ContainerStarted","Data":"275b266563413ed3c46cd15563e7db2c5075ab8c477f1b5cf74915fb4b05e39b"}
May 29 03:19:49.791575 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:49.791486    2559 generic.go:358] "Generic (PLEG): container finished" podID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerID="4325dd5eb8eea13d70d9f0f070c66909ced03396b31acf3c0639b8b9e1e99197" exitCode=0
May 29 03:19:49.791575 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:49.791560    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" event={"ID":"ac216ba2-d2f5-4d54-b527-c9dd88764cdf","Type":"ContainerDied","Data":"4325dd5eb8eea13d70d9f0f070c66909ced03396b31acf3c0639b8b9e1e99197"}
May 29 03:19:50.796002 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:50.795948    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" event={"ID":"ac216ba2-d2f5-4d54-b527-c9dd88764cdf","Type":"ContainerStarted","Data":"609d5c157204fb65f4e065137aec1edc9f4e5b7e76b2969d41d5d2ab931ccea0"}
May 29 03:19:55.811717 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:55.811681    2559 generic.go:358] "Generic (PLEG): container finished" podID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerID="609d5c157204fb65f4e065137aec1edc9f4e5b7e76b2969d41d5d2ab931ccea0" exitCode=0
May 29 03:19:55.812122 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:55.811748    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" event={"ID":"ac216ba2-d2f5-4d54-b527-c9dd88764cdf","Type":"ContainerDied","Data":"609d5c157204fb65f4e065137aec1edc9f4e5b7e76b2969d41d5d2ab931ccea0"}
May 29 03:19:56.817697 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:56.817657    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" event={"ID":"ac216ba2-d2f5-4d54-b527-c9dd88764cdf","Type":"ContainerStarted","Data":"7a1d12c3669985bcd4f7eb1bf8cf94d01f2d17112b5b74ad9515372664137825"}
May 29 03:19:56.817697 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:56.817698    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" event={"ID":"ac216ba2-d2f5-4d54-b527-c9dd88764cdf","Type":"ContainerStarted","Data":"957112d81ecbedf77245f0261328836c42e0a83eb619c77a12f36728b700347a"}
May 29 03:19:56.818274 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:56.817708    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" event={"ID":"ac216ba2-d2f5-4d54-b527-c9dd88764cdf","Type":"ContainerStarted","Data":"4af3e9045e6d0be5c16dcf928039eeb0054e6d5653e11b1cf66ba598ea6345b8"}
May 29 03:19:56.819804 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:56.819773    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:19:56.819940 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:56.819813    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:19:56.819940 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:56.819828    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:19:56.821223 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:56.821190    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.28:8816: connect: connection refused"
May 29 03:19:56.821345 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:56.821219    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused"
May 29 03:19:56.821345 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:56.821234    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused"
May 29 03:19:56.838913 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:56.838873    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" podStartSLOduration=13.838860879 podStartE2EDuration="13.838860879s" podCreationTimestamp="2026-05-29 03:19:43 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 03:19:56.837744027 +0000 UTC m=+1647.593059105" watchObservedRunningTime="2026-05-29 03:19:56.838860879 +0000 UTC m=+1647.594175943"
May 29 03:19:59.820539 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:59.820496    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused"
May 29 03:19:59.821106 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:59.820500    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.28:8816: connect: connection refused"
May 29 03:19:59.821298 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:19:59.821259    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused"
May 29 03:20:02.821215 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:20:02.821183    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:20:02.821599 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:20:02.821309    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:20:02.821699 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:20:02.821680    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:20:02.821754 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:20:02.821712    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:20:02.822094 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:20:02.822076    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:20:02.822197 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:20:02.822127    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:20:02.825167 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:20:02.825146    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:20:02.839572 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:20:02.839556    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:20:02.842148 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:20:02.842125    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:21:04.726874 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:04.726840    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"]
May 29 03:21:04.727339 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:04.727288    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="online" containerID="cri-o://4af3e9045e6d0be5c16dcf928039eeb0054e6d5653e11b1cf66ba598ea6345b8" gracePeriod=30
May 29 03:21:04.727411 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:04.727358    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="ui" containerID="cri-o://7a1d12c3669985bcd4f7eb1bf8cf94d01f2d17112b5b74ad9515372664137825" gracePeriod=30
May 29 03:21:04.727465 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:04.727437    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="offline" containerID="cri-o://957112d81ecbedf77245f0261328836c42e0a83eb619c77a12f36728b700347a" gracePeriod=30
May 29 03:21:04.902643 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:04.902606    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"]
May 29 03:21:04.903040 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:04.902942    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="registry" containerID="cri-o://dd2776e69f85668097bb4d369288416b42a5cd681278c7a40d51ada31e800f0d" gracePeriod=30
May 29 03:21:04.903206 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:04.903034    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="offline" containerID="cri-o://9a7bf818389c22039c5ead80e9f89bf50a4000f6a16d2139118afa1986ae17bf" gracePeriod=30
May 29 03:21:04.903206 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:04.903037    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="ui" containerID="cri-o://be85097d6dba776169f2fcd12df7f641ca8ade9807e4d8568240b281462cdc5e" gracePeriod=30
May 29 03:21:04.903206 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:04.903048    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="online" containerID="cri-o://6065da4dba7c9d8d35c8db52205dea642fe89e835d4f9a0c50378d8d37b0a134" gracePeriod=30
May 29 03:21:06.017160 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:06.017122    2559 generic.go:358] "Generic (PLEG): container finished" podID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerID="7a1d12c3669985bcd4f7eb1bf8cf94d01f2d17112b5b74ad9515372664137825" exitCode=0
May 29 03:21:06.017160 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:06.017155    2559 generic.go:358] "Generic (PLEG): container finished" podID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerID="4af3e9045e6d0be5c16dcf928039eeb0054e6d5653e11b1cf66ba598ea6345b8" exitCode=0
May 29 03:21:06.017613 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:06.017192    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" event={"ID":"ac216ba2-d2f5-4d54-b527-c9dd88764cdf","Type":"ContainerDied","Data":"7a1d12c3669985bcd4f7eb1bf8cf94d01f2d17112b5b74ad9515372664137825"}
May 29 03:21:06.017613 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:06.017228    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" event={"ID":"ac216ba2-d2f5-4d54-b527-c9dd88764cdf","Type":"ContainerDied","Data":"4af3e9045e6d0be5c16dcf928039eeb0054e6d5653e11b1cf66ba598ea6345b8"}
May 29 03:21:06.019454 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:06.019434    2559 generic.go:358] "Generic (PLEG): container finished" podID="00547dba-08f8-441a-8aff-e82990542301" containerID="be85097d6dba776169f2fcd12df7f641ca8ade9807e4d8568240b281462cdc5e" exitCode=0
May 29 03:21:06.019454 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:06.019452    2559 generic.go:358] "Generic (PLEG): container finished" podID="00547dba-08f8-441a-8aff-e82990542301" containerID="6065da4dba7c9d8d35c8db52205dea642fe89e835d4f9a0c50378d8d37b0a134" exitCode=0
May 29 03:21:06.019598 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:06.019501    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" event={"ID":"00547dba-08f8-441a-8aff-e82990542301","Type":"ContainerDied","Data":"be85097d6dba776169f2fcd12df7f641ca8ade9807e4d8568240b281462cdc5e"}
May 29 03:21:06.019598 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:06.019524    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" event={"ID":"00547dba-08f8-441a-8aff-e82990542301","Type":"ContainerDied","Data":"6065da4dba7c9d8d35c8db52205dea642fe89e835d4f9a0c50378d8d37b0a134"}
May 29 03:21:11.584628 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:11.584583    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused"
May 29 03:21:11.585021 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:11.584603    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused"
May 29 03:21:12.822628 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:12.822590    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused"
May 29 03:21:12.840177 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:12.840147    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused"
May 29 03:21:21.584109 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:21.584065    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused"
May 29 03:21:21.584109 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:21.584084    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused"
May 29 03:21:22.822576 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:22.822531    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused"
May 29 03:21:22.840019 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:22.839985    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused"
May 29 03:21:31.584782 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:31.584742    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused"
May 29 03:21:31.585419 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:31.584757    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused"
May 29 03:21:31.585419 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:31.584856    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:21:31.585419 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:31.584903    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:21:32.822024 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:32.821959    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused"
May 29 03:21:32.822442 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:32.822106    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:21:32.840126 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:32.840102    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused"
May 29 03:21:32.840208 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:32.840183    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:21:34.929070 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:34.929042    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:21:35.093266 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.093188    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grq9d\" (UniqueName: \"kubernetes.io/projected/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-kube-api-access-grq9d\") pod \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\" (UID: \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\") "
May 29 03:21:35.093266 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.093251    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-offline-tls\") pod \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\" (UID: \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\") "
May 29 03:21:35.093486 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.093296    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-online-tls\") pod \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\" (UID: \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\") "
May 29 03:21:35.093486 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.093328    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-feast-data\") pod \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\" (UID: \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\") "
May 29 03:21:35.093486 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.093344    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-ui-tls\") pod \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\" (UID: \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\") "
May 29 03:21:35.093486 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.093380    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-registry-tls\") pod \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\" (UID: \"ac216ba2-d2f5-4d54-b527-c9dd88764cdf\") "
May 29 03:21:35.093806 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.093773    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ac216ba2-d2f5-4d54-b527-c9dd88764cdf" (UID: "ac216ba2-d2f5-4d54-b527-c9dd88764cdf"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 03:21:35.093933 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.093819    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-feast-data" (OuterVolumeSpecName: "feast-data") pod "ac216ba2-d2f5-4d54-b527-c9dd88764cdf" (UID: "ac216ba2-d2f5-4d54-b527-c9dd88764cdf"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 03:21:35.095490 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.095468    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "ac216ba2-d2f5-4d54-b527-c9dd88764cdf" (UID: "ac216ba2-d2f5-4d54-b527-c9dd88764cdf"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 03:21:35.095593 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.095494    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-online-tls" (OuterVolumeSpecName: "online-tls") pod "ac216ba2-d2f5-4d54-b527-c9dd88764cdf" (UID: "ac216ba2-d2f5-4d54-b527-c9dd88764cdf"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 03:21:35.095593 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.095576    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-kube-api-access-grq9d" (OuterVolumeSpecName: "kube-api-access-grq9d") pod "ac216ba2-d2f5-4d54-b527-c9dd88764cdf" (UID: "ac216ba2-d2f5-4d54-b527-c9dd88764cdf"). InnerVolumeSpecName "kube-api-access-grq9d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 03:21:35.095681 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.095651    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "ac216ba2-d2f5-4d54-b527-c9dd88764cdf" (UID: "ac216ba2-d2f5-4d54-b527-c9dd88764cdf"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 03:21:35.105181 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.105151    2559 generic.go:358] "Generic (PLEG): container finished" podID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerID="957112d81ecbedf77245f0261328836c42e0a83eb619c77a12f36728b700347a" exitCode=137
May 29 03:21:35.105284 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.105223    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"
May 29 03:21:35.105284 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.105234    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" event={"ID":"ac216ba2-d2f5-4d54-b527-c9dd88764cdf","Type":"ContainerDied","Data":"957112d81ecbedf77245f0261328836c42e0a83eb619c77a12f36728b700347a"}
May 29 03:21:35.105396 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.105283    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8" event={"ID":"ac216ba2-d2f5-4d54-b527-c9dd88764cdf","Type":"ContainerDied","Data":"275b266563413ed3c46cd15563e7db2c5075ab8c477f1b5cf74915fb4b05e39b"}
May 29 03:21:35.105396 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.105305    2559 scope.go:117] "RemoveContainer" containerID="7a1d12c3669985bcd4f7eb1bf8cf94d01f2d17112b5b74ad9515372664137825"
May 29 03:21:35.108016 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.107993    2559 generic.go:358] "Generic (PLEG): container finished" podID="00547dba-08f8-441a-8aff-e82990542301" containerID="9a7bf818389c22039c5ead80e9f89bf50a4000f6a16d2139118afa1986ae17bf" exitCode=137
May 29 03:21:35.108016 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.108017    2559 generic.go:358] "Generic (PLEG): container finished" podID="00547dba-08f8-441a-8aff-e82990542301" containerID="dd2776e69f85668097bb4d369288416b42a5cd681278c7a40d51ada31e800f0d" exitCode=137
May 29 03:21:35.108160 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.108073    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" event={"ID":"00547dba-08f8-441a-8aff-e82990542301","Type":"ContainerDied","Data":"9a7bf818389c22039c5ead80e9f89bf50a4000f6a16d2139118afa1986ae17bf"}
May 29 03:21:35.108160 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.108109    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" event={"ID":"00547dba-08f8-441a-8aff-e82990542301","Type":"ContainerDied","Data":"dd2776e69f85668097bb4d369288416b42a5cd681278c7a40d51ada31e800f0d"}
May 29 03:21:35.113995 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.113954    2559 scope.go:117] "RemoveContainer" containerID="957112d81ecbedf77245f0261328836c42e0a83eb619c77a12f36728b700347a"
May 29 03:21:35.120660 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.120630    2559 scope.go:117] "RemoveContainer" containerID="4af3e9045e6d0be5c16dcf928039eeb0054e6d5653e11b1cf66ba598ea6345b8"
May 29 03:21:35.125884 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.125856    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"]
May 29 03:21:35.127709 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.127692    2559 scope.go:117] "RemoveContainer" containerID="609d5c157204fb65f4e065137aec1edc9f4e5b7e76b2969d41d5d2ab931ccea0"
May 29 03:21:35.129135 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.129116    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wgbf8"]
May 29 03:21:35.134646 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.134632    2559 scope.go:117] "RemoveContainer" containerID="4325dd5eb8eea13d70d9f0f070c66909ced03396b31acf3c0639b8b9e1e99197"
May 29 03:21:35.141100 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.141086    2559 scope.go:117] "RemoveContainer" containerID="7a1d12c3669985bcd4f7eb1bf8cf94d01f2d17112b5b74ad9515372664137825"
May 29 03:21:35.141335 ip-10-0-137-122 kubenswrapper[2559]: E0529 03:21:35.141319    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a1d12c3669985bcd4f7eb1bf8cf94d01f2d17112b5b74ad9515372664137825\": container with ID starting with 7a1d12c3669985bcd4f7eb1bf8cf94d01f2d17112b5b74ad9515372664137825 not found: ID does not exist" containerID="7a1d12c3669985bcd4f7eb1bf8cf94d01f2d17112b5b74ad9515372664137825"
May 29 03:21:35.141379 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.141344    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a1d12c3669985bcd4f7eb1bf8cf94d01f2d17112b5b74ad9515372664137825"} err="failed to get container status \"7a1d12c3669985bcd4f7eb1bf8cf94d01f2d17112b5b74ad9515372664137825\": rpc error: code = NotFound desc = could not find container \"7a1d12c3669985bcd4f7eb1bf8cf94d01f2d17112b5b74ad9515372664137825\": container with ID starting with 7a1d12c3669985bcd4f7eb1bf8cf94d01f2d17112b5b74ad9515372664137825 not found: ID does not exist"
May 29 03:21:35.141379 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.141361    2559 scope.go:117] "RemoveContainer" containerID="957112d81ecbedf77245f0261328836c42e0a83eb619c77a12f36728b700347a"
May 29 03:21:35.141576 ip-10-0-137-122 kubenswrapper[2559]: E0529 03:21:35.141562    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"957112d81ecbedf77245f0261328836c42e0a83eb619c77a12f36728b700347a\": container with ID starting with 957112d81ecbedf77245f0261328836c42e0a83eb619c77a12f36728b700347a not found: ID does not exist" containerID="957112d81ecbedf77245f0261328836c42e0a83eb619c77a12f36728b700347a"
May 29 03:21:35.141623 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.141578    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"957112d81ecbedf77245f0261328836c42e0a83eb619c77a12f36728b700347a"} err="failed to get container status \"957112d81ecbedf77245f0261328836c42e0a83eb619c77a12f36728b700347a\": rpc error: code = NotFound desc = could not find container \"957112d81ecbedf77245f0261328836c42e0a83eb619c77a12f36728b700347a\": container with ID starting with 957112d81ecbedf77245f0261328836c42e0a83eb619c77a12f36728b700347a not found: ID does not exist"
May 29 03:21:35.141623 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.141589    2559 scope.go:117] "RemoveContainer" containerID="4af3e9045e6d0be5c16dcf928039eeb0054e6d5653e11b1cf66ba598ea6345b8"
May 29 03:21:35.141801 ip-10-0-137-122 kubenswrapper[2559]: E0529 03:21:35.141783    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4af3e9045e6d0be5c16dcf928039eeb0054e6d5653e11b1cf66ba598ea6345b8\": container with ID starting with 4af3e9045e6d0be5c16dcf928039eeb0054e6d5653e11b1cf66ba598ea6345b8 not found: ID does not exist" containerID="4af3e9045e6d0be5c16dcf928039eeb0054e6d5653e11b1cf66ba598ea6345b8"
May 29 03:21:35.141842 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.141808    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4af3e9045e6d0be5c16dcf928039eeb0054e6d5653e11b1cf66ba598ea6345b8"} err="failed to get container status \"4af3e9045e6d0be5c16dcf928039eeb0054e6d5653e11b1cf66ba598ea6345b8\": rpc error: code = NotFound desc = could not find container \"4af3e9045e6d0be5c16dcf928039eeb0054e6d5653e11b1cf66ba598ea6345b8\": container with ID starting with 4af3e9045e6d0be5c16dcf928039eeb0054e6d5653e11b1cf66ba598ea6345b8 not found: ID does not exist"
May 29 03:21:35.141842 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.141829    2559 scope.go:117] "RemoveContainer" containerID="609d5c157204fb65f4e065137aec1edc9f4e5b7e76b2969d41d5d2ab931ccea0"
May 29 03:21:35.142070 ip-10-0-137-122 kubenswrapper[2559]: E0529 03:21:35.142055    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"609d5c157204fb65f4e065137aec1edc9f4e5b7e76b2969d41d5d2ab931ccea0\": container with ID starting with 609d5c157204fb65f4e065137aec1edc9f4e5b7e76b2969d41d5d2ab931ccea0 not found: ID does not exist" containerID="609d5c157204fb65f4e065137aec1edc9f4e5b7e76b2969d41d5d2ab931ccea0"
May 29 03:21:35.142117 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.142073    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"609d5c157204fb65f4e065137aec1edc9f4e5b7e76b2969d41d5d2ab931ccea0"} err="failed to get container status \"609d5c157204fb65f4e065137aec1edc9f4e5b7e76b2969d41d5d2ab931ccea0\": rpc error: code = NotFound desc = could not find container \"609d5c157204fb65f4e065137aec1edc9f4e5b7e76b2969d41d5d2ab931ccea0\": container with ID starting with 609d5c157204fb65f4e065137aec1edc9f4e5b7e76b2969d41d5d2ab931ccea0 not found: ID does not exist"
May 29 03:21:35.142117 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.142086    2559 scope.go:117] "RemoveContainer" containerID="4325dd5eb8eea13d70d9f0f070c66909ced03396b31acf3c0639b8b9e1e99197"
May 29 03:21:35.142267 ip-10-0-137-122 kubenswrapper[2559]: E0529 03:21:35.142252    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4325dd5eb8eea13d70d9f0f070c66909ced03396b31acf3c0639b8b9e1e99197\": container with ID starting with 4325dd5eb8eea13d70d9f0f070c66909ced03396b31acf3c0639b8b9e1e99197 not found: ID does not exist" containerID="4325dd5eb8eea13d70d9f0f070c66909ced03396b31acf3c0639b8b9e1e99197"
May 29 03:21:35.142317 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.142270    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4325dd5eb8eea13d70d9f0f070c66909ced03396b31acf3c0639b8b9e1e99197"} err="failed to get container status \"4325dd5eb8eea13d70d9f0f070c66909ced03396b31acf3c0639b8b9e1e99197\": rpc error: code = NotFound desc = could not find container \"4325dd5eb8eea13d70d9f0f070c66909ced03396b31acf3c0639b8b9e1e99197\": container with ID starting with 4325dd5eb8eea13d70d9f0f070c66909ced03396b31acf3c0639b8b9e1e99197 not found: ID does not exist"
May 29 03:21:35.194808 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.194788    2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-registry-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:21:35.194808 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.194810    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grq9d\" (UniqueName: \"kubernetes.io/projected/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-kube-api-access-grq9d\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:21:35.194925 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.194820    2559 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-offline-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:21:35.194925 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.194831    2559 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-online-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:21:35.194925 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.194840    2559 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-feast-data\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:21:35.194925 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.194848    2559 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ac216ba2-d2f5-4d54-b527-c9dd88764cdf-ui-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:21:35.544623 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.544602    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:21:35.699062 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.698990    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-registry-tls\") pod \"00547dba-08f8-441a-8aff-e82990542301\" (UID: \"00547dba-08f8-441a-8aff-e82990542301\") "
May 29 03:21:35.699062 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.699047    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-offline-tls\") pod \"00547dba-08f8-441a-8aff-e82990542301\" (UID: \"00547dba-08f8-441a-8aff-e82990542301\") "
May 29 03:21:35.699249 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.699074    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-online-tls\") pod \"00547dba-08f8-441a-8aff-e82990542301\" (UID: \"00547dba-08f8-441a-8aff-e82990542301\") "
May 29 03:21:35.699249 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.699105    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-ui-tls\") pod \"00547dba-08f8-441a-8aff-e82990542301\" (UID: \"00547dba-08f8-441a-8aff-e82990542301\") "
May 29 03:21:35.699249 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.699150    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/00547dba-08f8-441a-8aff-e82990542301-feast-data\") pod \"00547dba-08f8-441a-8aff-e82990542301\" (UID: \"00547dba-08f8-441a-8aff-e82990542301\") "
May 29 03:21:35.699249 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.699196    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4c9qt\" (UniqueName: \"kubernetes.io/projected/00547dba-08f8-441a-8aff-e82990542301-kube-api-access-4c9qt\") pod \"00547dba-08f8-441a-8aff-e82990542301\" (UID: \"00547dba-08f8-441a-8aff-e82990542301\") "
May 29 03:21:35.699756 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.699705    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00547dba-08f8-441a-8aff-e82990542301-feast-data" (OuterVolumeSpecName: "feast-data") pod "00547dba-08f8-441a-8aff-e82990542301" (UID: "00547dba-08f8-441a-8aff-e82990542301"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 03:21:35.701418 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.701394    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00547dba-08f8-441a-8aff-e82990542301-kube-api-access-4c9qt" (OuterVolumeSpecName: "kube-api-access-4c9qt") pod "00547dba-08f8-441a-8aff-e82990542301" (UID: "00547dba-08f8-441a-8aff-e82990542301"). InnerVolumeSpecName "kube-api-access-4c9qt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 03:21:35.701715 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.701692    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "00547dba-08f8-441a-8aff-e82990542301" (UID: "00547dba-08f8-441a-8aff-e82990542301"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 03:21:35.701807 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.701712    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "00547dba-08f8-441a-8aff-e82990542301" (UID: "00547dba-08f8-441a-8aff-e82990542301"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 03:21:35.701807 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.701721    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-online-tls" (OuterVolumeSpecName: "online-tls") pod "00547dba-08f8-441a-8aff-e82990542301" (UID: "00547dba-08f8-441a-8aff-e82990542301"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 03:21:35.701807 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.701737    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "00547dba-08f8-441a-8aff-e82990542301" (UID: "00547dba-08f8-441a-8aff-e82990542301"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 03:21:35.800573 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.800549    2559 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/00547dba-08f8-441a-8aff-e82990542301-feast-data\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:21:35.800573 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.800572    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4c9qt\" (UniqueName: \"kubernetes.io/projected/00547dba-08f8-441a-8aff-e82990542301-kube-api-access-4c9qt\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:21:35.800728 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.800584    2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-registry-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:21:35.800728 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.800592    2559 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-offline-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:21:35.800728 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.800600    2559 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-online-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:21:35.800728 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.800608    2559 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/00547dba-08f8-441a-8aff-e82990542301-ui-tls\") on node \"ip-10-0-137-122.ec2.internal\" DevicePath \"\""
May 29 03:21:35.848661 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:35.848627    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" path="/var/lib/kubelet/pods/ac216ba2-d2f5-4d54-b527-c9dd88764cdf/volumes"
May 29 03:21:36.113335 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:36.113304    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km" event={"ID":"00547dba-08f8-441a-8aff-e82990542301","Type":"ContainerDied","Data":"48cf2916aa3227cb99abbe6cec1f339607f808fddae4598fe1da85227ba77942"}
May 29 03:21:36.113711 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:36.113346    2559 scope.go:117] "RemoveContainer" containerID="be85097d6dba776169f2fcd12df7f641ca8ade9807e4d8568240b281462cdc5e"
May 29 03:21:36.113711 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:36.113349    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"
May 29 03:21:36.121260 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:36.121242    2559 scope.go:117] "RemoveContainer" containerID="9a7bf818389c22039c5ead80e9f89bf50a4000f6a16d2139118afa1986ae17bf"
May 29 03:21:36.127951 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:36.127937    2559 scope.go:117] "RemoveContainer" containerID="6065da4dba7c9d8d35c8db52205dea642fe89e835d4f9a0c50378d8d37b0a134"
May 29 03:21:36.131282 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:36.131261    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"]
May 29 03:21:36.135864 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:36.135821    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-858km"]
May 29 03:21:36.136043 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:36.136016    2559 scope.go:117] "RemoveContainer" containerID="dd2776e69f85668097bb4d369288416b42a5cd681278c7a40d51ada31e800f0d"
May 29 03:21:36.143016 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:36.143000    2559 scope.go:117] "RemoveContainer" containerID="a80e30027f8bccd20ec2a5396fa9b37a0ede2ce0bc6ae5951ce9e7628073e246"
May 29 03:21:36.149600 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:36.149587    2559 scope.go:117] "RemoveContainer" containerID="92fd35e0cb3fc7d0c9155a7e56b942a4b08416e8ff560089961b2f16b67f383b"
May 29 03:21:37.849092 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:21:37.849061    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00547dba-08f8-441a-8aff-e82990542301" path="/var/lib/kubelet/pods/00547dba-08f8-441a-8aff-e82990542301/volumes"
May 29 03:22:15.267015 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:15.266983    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hp2fn_75bf9ff1-3a9f-4140-9ba2-bac8a5b588c1/global-pull-secret-syncer/0.log"
May 29 03:22:15.404866 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:15.404840    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-h2vvt_deaa9f6f-56b4-4875-85fc-a2396ad84029/konnectivity-agent/0.log"
May 29 03:22:15.520868 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:15.520791    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-122.ec2.internal_fcf3ae000999ff406cf9a785430af920/haproxy/0.log"
May 29 03:22:18.808015 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:18.807930    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f8649618-fbf6-4f7b-9281-a1942009267b/alertmanager/0.log"
May 29 03:22:18.832473 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:18.832405    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f8649618-fbf6-4f7b-9281-a1942009267b/config-reloader/0.log"
May 29 03:22:18.859498 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:18.859479    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f8649618-fbf6-4f7b-9281-a1942009267b/kube-rbac-proxy-web/0.log"
May 29 03:22:18.885955 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:18.885924    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f8649618-fbf6-4f7b-9281-a1942009267b/kube-rbac-proxy/0.log"
May 29 03:22:18.909893 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:18.909869    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f8649618-fbf6-4f7b-9281-a1942009267b/kube-rbac-proxy-metric/0.log"
May 29 03:22:18.936554 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:18.936535    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f8649618-fbf6-4f7b-9281-a1942009267b/prom-label-proxy/0.log"
May 29 03:22:18.962577 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:18.962554    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f8649618-fbf6-4f7b-9281-a1942009267b/init-config-reloader/0.log"
May 29 03:22:19.173406 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:19.173374    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5jltk_f09e25bb-880d-47c5-aef4-aa43f3ef48b9/node-exporter/0.log"
May 29 03:22:19.195313 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:19.195292    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5jltk_f09e25bb-880d-47c5-aef4-aa43f3ef48b9/kube-rbac-proxy/0.log"
May 29 03:22:19.219121 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:19.219102    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5jltk_f09e25bb-880d-47c5-aef4-aa43f3ef48b9/init-textfile/0.log"
May 29 03:22:19.413657 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:19.413631    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-b2kdq_97a3a5c7-f634-4f14-9961-1970e366b867/kube-rbac-proxy-main/0.log"
May 29 03:22:19.439903 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:19.439835    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-b2kdq_97a3a5c7-f634-4f14-9961-1970e366b867/kube-rbac-proxy-self/0.log"
May 29 03:22:19.470467 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:19.470446    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-b2kdq_97a3a5c7-f634-4f14-9961-1970e366b867/openshift-state-metrics/0.log"
May 29 03:22:19.532417 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:19.532396    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e73ab701-75d2-4f51-9e02-bb1145342d8a/prometheus/0.log"
May 29 03:22:19.556581 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:19.556561    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e73ab701-75d2-4f51-9e02-bb1145342d8a/config-reloader/0.log"
May 29 03:22:19.589497 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:19.589479    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e73ab701-75d2-4f51-9e02-bb1145342d8a/thanos-sidecar/0.log"
May 29 03:22:19.619956 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:19.619932    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e73ab701-75d2-4f51-9e02-bb1145342d8a/kube-rbac-proxy-web/0.log"
May 29 03:22:19.650211 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:19.650194    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e73ab701-75d2-4f51-9e02-bb1145342d8a/kube-rbac-proxy/0.log"
May 29 03:22:19.675153 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:19.675132    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e73ab701-75d2-4f51-9e02-bb1145342d8a/kube-rbac-proxy-thanos/0.log"
May 29 03:22:19.706040 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:19.705987    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e73ab701-75d2-4f51-9e02-bb1145342d8a/init-config-reloader/0.log"
May 29 03:22:19.743182 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:19.743163    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7779bbdd78-vgtrr_d948ccb0-1cad-47d5-85ea-69f2c3189425/prometheus-operator/0.log"
May 29 03:22:19.764182 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:19.764164    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7779bbdd78-vgtrr_d948ccb0-1cad-47d5-85ea-69f2c3189425/kube-rbac-proxy/0.log"
May 29 03:22:19.791006 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:19.790987    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6c4bbc67b7-t6s4g_76fded48-86f5-4d63-9184-0a8cd59e77ac/prometheus-operator-admission-webhook/0.log"
May 29 03:22:21.511014 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.510980    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-89vx4/perf-node-gather-daemonset-w6txm"]
May 29 03:22:21.511526 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511445    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="offline"
May 29 03:22:21.511526 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511464    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="offline"
May 29 03:22:21.511526 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511476    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="ui"
May 29 03:22:21.511526 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511485    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="ui"
May 29 03:22:21.511526 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511497    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="feast-apply"
May 29 03:22:21.511526 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511505    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="feast-apply"
May 29 03:22:21.511526 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511518    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="feast-init"
May 29 03:22:21.511526 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511527    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="feast-init"
May 29 03:22:21.511931 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511542    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="online"
May 29 03:22:21.511931 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511550    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="online"
May 29 03:22:21.511931 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511560    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="offline"
May 29 03:22:21.511931 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511568    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="offline"
May 29 03:22:21.511931 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511580    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="online"
May 29 03:22:21.511931 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511589    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="online"
May 29 03:22:21.511931 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511604    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="ui"
May 29 03:22:21.511931 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511612    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="ui"
May 29 03:22:21.511931 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511627    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="feast-init"
May 29 03:22:21.511931 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511635    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="feast-init"
May 29 03:22:21.511931 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511643    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="feast-apply"
May 29 03:22:21.511931 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511652    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="feast-apply"
May 29 03:22:21.511931 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511663    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="registry"
May 29 03:22:21.511931 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511671    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="registry"
May 29 03:22:21.511931 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511742    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="ui"
May 29 03:22:21.511931 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511757    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="ui"
May 29 03:22:21.511931 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511768    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="registry"
May 29 03:22:21.511931 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511777    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="online"
May 29 03:22:21.511931 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511787    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="00547dba-08f8-441a-8aff-e82990542301" containerName="offline"
May 29 03:22:21.511931 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511799    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="online"
May 29 03:22:21.511931 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.511808    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="ac216ba2-d2f5-4d54-b527-c9dd88764cdf" containerName="offline"
May 29 03:22:21.515060 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.515042    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-89vx4/perf-node-gather-daemonset-w6txm"
May 29 03:22:21.521088 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.521041    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-89vx4\"/\"openshift-service-ca.crt\""
May 29 03:22:21.521184 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.521150    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-89vx4\"/\"kube-root-ca.crt\""
May 29 03:22:21.521429 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.521412    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-89vx4\"/\"default-dockercfg-cqj8h\""
May 29 03:22:21.526414 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.526394    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-89vx4/perf-node-gather-daemonset-w6txm"]
May 29 03:22:21.638108 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.638086    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b-lib-modules\") pod \"perf-node-gather-daemonset-w6txm\" (UID: \"f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b\") " pod="openshift-must-gather-89vx4/perf-node-gather-daemonset-w6txm"
May 29 03:22:21.638230 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.638124    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqkqz\" (UniqueName: \"kubernetes.io/projected/f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b-kube-api-access-hqkqz\") pod \"perf-node-gather-daemonset-w6txm\" (UID: \"f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b\") " pod="openshift-must-gather-89vx4/perf-node-gather-daemonset-w6txm"
May 29 03:22:21.638230 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.638176    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b-proc\") pod \"perf-node-gather-daemonset-w6txm\" (UID: \"f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b\") " pod="openshift-must-gather-89vx4/perf-node-gather-daemonset-w6txm"
May 29 03:22:21.638230 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.638209    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b-sys\") pod \"perf-node-gather-daemonset-w6txm\" (UID: \"f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b\") " pod="openshift-must-gather-89vx4/perf-node-gather-daemonset-w6txm"
May 29 03:22:21.638230 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.638227    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b-podres\") pod \"perf-node-gather-daemonset-w6txm\" (UID: \"f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b\") " pod="openshift-must-gather-89vx4/perf-node-gather-daemonset-w6txm"
May 29 03:22:21.739113 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.739088    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b-proc\") pod \"perf-node-gather-daemonset-w6txm\" (UID: \"f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b\") " pod="openshift-must-gather-89vx4/perf-node-gather-daemonset-w6txm"
May 29 03:22:21.739211 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.739118    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b-sys\") pod \"perf-node-gather-daemonset-w6txm\" (UID: \"f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b\") " pod="openshift-must-gather-89vx4/perf-node-gather-daemonset-w6txm"
May 29 03:22:21.739211 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.739136    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b-podres\") pod \"perf-node-gather-daemonset-w6txm\" (UID: \"f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b\") " pod="openshift-must-gather-89vx4/perf-node-gather-daemonset-w6txm"
May 29 03:22:21.739211 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.739178    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b-lib-modules\") pod \"perf-node-gather-daemonset-w6txm\" (UID: \"f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b\") " pod="openshift-must-gather-89vx4/perf-node-gather-daemonset-w6txm"
May 29 03:22:21.739211 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.739200    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b-proc\") pod \"perf-node-gather-daemonset-w6txm\" (UID: \"f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b\") " pod="openshift-must-gather-89vx4/perf-node-gather-daemonset-w6txm"
May 29 03:22:21.739351 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.739214    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b-sys\") pod \"perf-node-gather-daemonset-w6txm\" (UID: \"f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b\") " pod="openshift-must-gather-89vx4/perf-node-gather-daemonset-w6txm"
May 29 03:22:21.739351 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.739223    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hqkqz\" (UniqueName: \"kubernetes.io/projected/f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b-kube-api-access-hqkqz\") pod \"perf-node-gather-daemonset-w6txm\" (UID: \"f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b\") " pod="openshift-must-gather-89vx4/perf-node-gather-daemonset-w6txm"
May 29 03:22:21.739351 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.739299    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b-podres\") pod \"perf-node-gather-daemonset-w6txm\" (UID: \"f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b\") " pod="openshift-must-gather-89vx4/perf-node-gather-daemonset-w6txm"
May 29 03:22:21.739351 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.739328    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b-lib-modules\") pod \"perf-node-gather-daemonset-w6txm\" (UID: \"f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b\") " pod="openshift-must-gather-89vx4/perf-node-gather-daemonset-w6txm"
May 29 03:22:21.747139 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.747116    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqkqz\" (UniqueName: \"kubernetes.io/projected/f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b-kube-api-access-hqkqz\") pod \"perf-node-gather-daemonset-w6txm\" (UID: \"f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b\") " pod="openshift-must-gather-89vx4/perf-node-gather-daemonset-w6txm"
May 29 03:22:21.824918 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.824855    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-89vx4/perf-node-gather-daemonset-w6txm"
May 29 03:22:21.948784 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.948755    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-89vx4/perf-node-gather-daemonset-w6txm"]
May 29 03:22:21.951852 ip-10-0-137-122 kubenswrapper[2559]: W0529 03:22:21.951821    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf52d1f7a_6ed7_4fe9_bc8a_1fe7b6f4ce9b.slice/crio-195f57d10495de6a45c7897654650ba8171458784df40ef2d69d94cc46565a52 WatchSource:0}: Error finding container 195f57d10495de6a45c7897654650ba8171458784df40ef2d69d94cc46565a52: Status 404 returned error can't find the container with id 195f57d10495de6a45c7897654650ba8171458784df40ef2d69d94cc46565a52
May 29 03:22:21.953396 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:21.953381    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 29 03:22:22.130093 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:22.130072    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5cf8c89d64-mtt7h_300ba3a2-0ab1-4944-b6f6-03922b888d1b/volume-data-source-validator/0.log"
May 29 03:22:22.246275 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:22.246242    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-89vx4/perf-node-gather-daemonset-w6txm" event={"ID":"f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b","Type":"ContainerStarted","Data":"03ecb44959e396edb62ec72853880e15dab3c7072c3df64c1ff24d2df402782f"}
May 29 03:22:22.246450 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:22.246290    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-89vx4/perf-node-gather-daemonset-w6txm" event={"ID":"f52d1f7a-6ed7-4fe9-bc8a-1fe7b6f4ce9b","Type":"ContainerStarted","Data":"195f57d10495de6a45c7897654650ba8171458784df40ef2d69d94cc46565a52"}
May 29 03:22:22.246450 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:22.246326    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-89vx4/perf-node-gather-daemonset-w6txm"
May 29 03:22:22.260795 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:22.260757    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-89vx4/perf-node-gather-daemonset-w6txm" podStartSLOduration=1.260745799 podStartE2EDuration="1.260745799s" podCreationTimestamp="2026-05-29 03:22:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 03:22:22.259905338 +0000 UTC m=+1793.015220412" watchObservedRunningTime="2026-05-29 03:22:22.260745799 +0000 UTC m=+1793.016060862"
May 29 03:22:22.802471 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:22.802425    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9c6sj_e7cca70d-3eb6-4647-9508-894b981687eb/dns/0.log"
May 29 03:22:22.824575 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:22.824556    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9c6sj_e7cca70d-3eb6-4647-9508-894b981687eb/kube-rbac-proxy/0.log"
May 29 03:22:22.931420 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:22.931392    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wz44t_846260de-5ed8-4c80-ad40-8480a09eddd2/dns-node-resolver/0.log"
May 29 03:22:23.369503 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:23.369471    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-92nfp_235d8ad4-77e9-477a-b8d4-4edc46151e4c/node-ca/0.log"
May 29 03:22:24.448541 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:24.448519    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bzt86_e5ebef60-aa2e-4a66-ad59-307085eae08c/serve-healthcheck-canary/0.log"
May 29 03:22:24.821383 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:24.821307    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9thz9_503df8ce-8b42-4e8e-a59a-92b9a68b979d/kube-rbac-proxy/0.log"
May 29 03:22:24.843070 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:24.843047    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9thz9_503df8ce-8b42-4e8e-a59a-92b9a68b979d/exporter/0.log"
May 29 03:22:24.865507 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:24.865488    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9thz9_503df8ce-8b42-4e8e-a59a-92b9a68b979d/extractor/0.log"
May 29 03:22:28.258434 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:28.258405    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-89vx4/perf-node-gather-daemonset-w6txm"
May 29 03:22:29.234539 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:29.234506    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-xgkbz_a79bccd9-301d-4bfe-add6-f2a150ae2366/migrator/0.log"
May 29 03:22:29.256618 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:29.256596    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-xgkbz_a79bccd9-301d-4bfe-add6-f2a150ae2366/graceful-termination/0.log"
May 29 03:22:29.875333 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:29.875303    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psj9n_d9a08de9-bea6-4561-8eae-62421b7584a9/ovn-acl-logging/0.log"
May 29 03:22:29.876802 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:29.876783    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psj9n_d9a08de9-bea6-4561-8eae-62421b7584a9/ovn-acl-logging/0.log"
May 29 03:22:30.298005 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:30.297905    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-7fhjp_532da6c4-ac98-4477-88e3-021b3a046918/kube-multus/0.log"
May 29 03:22:30.320053 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:30.320032    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-22clf_cccb57a5-0894-4319-9eeb-2459f94a7a17/kube-multus-additional-cni-plugins/0.log"
May 29 03:22:30.340299 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:30.340280    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-22clf_cccb57a5-0894-4319-9eeb-2459f94a7a17/egress-router-binary-copy/0.log"
May 29 03:22:30.360809 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:30.360787    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-22clf_cccb57a5-0894-4319-9eeb-2459f94a7a17/cni-plugins/0.log"
May 29 03:22:30.381870 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:30.381850    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-22clf_cccb57a5-0894-4319-9eeb-2459f94a7a17/bond-cni-plugin/0.log"
May 29 03:22:30.402577 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:30.402555    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-22clf_cccb57a5-0894-4319-9eeb-2459f94a7a17/routeoverride-cni/0.log"
May 29 03:22:30.423470 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:30.423450    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-22clf_cccb57a5-0894-4319-9eeb-2459f94a7a17/whereabouts-cni-bincopy/0.log"
May 29 03:22:30.443859 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:30.443844    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-22clf_cccb57a5-0894-4319-9eeb-2459f94a7a17/whereabouts-cni/0.log"
May 29 03:22:30.995732 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:30.995702    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-r4cd8_757d1101-4ef1-451f-b984-012545775d40/network-metrics-daemon/0.log"
May 29 03:22:31.017491 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:31.017460    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-r4cd8_757d1101-4ef1-451f-b984-012545775d40/kube-rbac-proxy/0.log"
May 29 03:22:32.189741 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:32.189710    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psj9n_d9a08de9-bea6-4561-8eae-62421b7584a9/ovn-controller/0.log"
May 29 03:22:32.210244 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:32.210222    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psj9n_d9a08de9-bea6-4561-8eae-62421b7584a9/ovn-acl-logging/0.log"
May 29 03:22:32.226489 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:32.226469    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psj9n_d9a08de9-bea6-4561-8eae-62421b7584a9/ovn-acl-logging/1.log"
May 29 03:22:32.248171 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:32.248147    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psj9n_d9a08de9-bea6-4561-8eae-62421b7584a9/kube-rbac-proxy-node/0.log"
May 29 03:22:32.273906 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:32.273880    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psj9n_d9a08de9-bea6-4561-8eae-62421b7584a9/kube-rbac-proxy-ovn-metrics/0.log"
May 29 03:22:32.300551 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:32.300520    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psj9n_d9a08de9-bea6-4561-8eae-62421b7584a9/northd/0.log"
May 29 03:22:32.321550 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:32.321529    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psj9n_d9a08de9-bea6-4561-8eae-62421b7584a9/nbdb/0.log"
May 29 03:22:32.342924 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:32.342902    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psj9n_d9a08de9-bea6-4561-8eae-62421b7584a9/sbdb/0.log"
May 29 03:22:32.496380 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:32.496306    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psj9n_d9a08de9-bea6-4561-8eae-62421b7584a9/ovnkube-controller/0.log"
May 29 03:22:33.844144 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:33.844116    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-fnzpl_902df08f-86be-4b22-8929-a7087c7b2c9a/network-check-target-container/0.log"
May 29 03:22:34.826491 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:34.826408    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-zb5lm_32a9fc68-65c5-42d9-9121-6fa454c910f4/iptables-alerter/0.log"
May 29 03:22:35.430550 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:35.430521    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-c8wpt_aa6f0c45-98c6-473a-af3b-51b04f61b1c9/tuned/0.log"
May 29 03:22:37.025123 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:37.025094    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-697587599-x2nj4_b13aaac1-8719-4128-ad12-9e0463049b28/cluster-samples-operator/0.log"
May 29 03:22:37.042756 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:37.042732    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-697587599-x2nj4_b13aaac1-8719-4128-ad12-9e0463049b28/cluster-samples-operator-watch/0.log"
May 29 03:22:37.946104 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:37.946072    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca-operator_service-ca-operator-57b5b68b8-z2zvs_d897b3db-7602-4711-96e7-b21b92e4b006/service-ca-operator/1.log"
May 29 03:22:37.947625 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:37.947596    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca-operator_service-ca-operator-57b5b68b8-z2zvs_d897b3db-7602-4711-96e7-b21b92e4b006/service-ca-operator/0.log"
May 29 03:22:38.311174 ip-10-0-137-122 kubenswrapper[2559]: I0529 03:22:38.311100    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca_service-ca-67d667c7d-lsklf_62b9435a-1196-4653-b535-d4ed06119ff6/service-ca-controller/0.log"