May 28 00:53:20.043930 ip-10-0-131-73 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
May 28 00:53:20.043944 ip-10-0-131-73 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
May 28 00:53:20.043954 ip-10-0-131-73 systemd[1]: kubelet.service: Failed with result 'resources'.
May 28 00:53:20.044243 ip-10-0-131-73 systemd[1]: Failed to start Kubernetes Kubelet.
May 28 00:53:29.174218 ip-10-0-131-73 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
May 28 00:53:29.174233 ip-10-0-131-73 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 5a35359fe76b42de8e501e7945701491 --
May 28 00:55:40.587733 ip-10-0-131-73 systemd[1]: Starting Kubernetes Kubelet...
May 28 00:55:41.087292 ip-10-0-131-73 kubenswrapper[2550]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 00:55:41.087292 ip-10-0-131-73 kubenswrapper[2550]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 28 00:55:41.087292 ip-10-0-131-73 kubenswrapper[2550]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 00:55:41.087292 ip-10-0-131-73 kubenswrapper[2550]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 28 00:55:41.087292 ip-10-0-131-73 kubenswrapper[2550]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 00:55:41.089260 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.089175    2550 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 28 00:55:41.092396 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092381    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 00:55:41.092396 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092396    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 00:55:41.092474 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092400    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 00:55:41.092474 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092404    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 00:55:41.092474 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092407    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 00:55:41.092474 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092423    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 00:55:41.092474 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092426    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 00:55:41.092474 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092430    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 00:55:41.092474 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092432    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 00:55:41.092474 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092435    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 00:55:41.092474 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092438    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 00:55:41.092474 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092441    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 00:55:41.092474 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092444    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 00:55:41.092474 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092446    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 00:55:41.092474 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092449    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 00:55:41.092474 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092452    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 00:55:41.092474 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092454    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 00:55:41.092474 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092462    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 00:55:41.092474 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092465    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 00:55:41.092474 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092468    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 00:55:41.092474 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092471    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 00:55:41.092931 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092473    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 00:55:41.092931 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092476    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 00:55:41.092931 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092479    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 00:55:41.092931 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092482    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 00:55:41.092931 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092484    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 00:55:41.092931 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092487    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 00:55:41.092931 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092490    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 00:55:41.092931 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092492    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 00:55:41.092931 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092495    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 00:55:41.092931 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092498    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 00:55:41.092931 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092500    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 00:55:41.092931 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092503    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 00:55:41.092931 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092505    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 00:55:41.092931 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092508    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 00:55:41.092931 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092510    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 00:55:41.092931 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092514    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 00:55:41.092931 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092516    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 00:55:41.092931 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092526    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 00:55:41.092931 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092529    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 00:55:41.092931 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092531    2550 feature_gate.go:328] unrecognized feature gate: Example2
May 28 00:55:41.093438 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092534    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 00:55:41.093438 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092537    2550 feature_gate.go:328] unrecognized feature gate: Example
May 28 00:55:41.093438 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092540    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 00:55:41.093438 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092542    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 00:55:41.093438 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092545    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 00:55:41.093438 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092547    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 00:55:41.093438 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092549    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 00:55:41.093438 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092552    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 00:55:41.093438 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092555    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 00:55:41.093438 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092557    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 00:55:41.093438 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092560    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 00:55:41.093438 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092562    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 00:55:41.093438 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092565    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 00:55:41.093438 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092569    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 00:55:41.093438 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092571    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 00:55:41.093438 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092574    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 00:55:41.093438 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092577    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 00:55:41.093438 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092580    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 00:55:41.093438 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092582    2550 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 00:55:41.093438 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092585    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 00:55:41.093939 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092587    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 00:55:41.093939 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092590    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 00:55:41.093939 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092592    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 00:55:41.093939 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092595    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 00:55:41.093939 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092597    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 00:55:41.093939 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092607    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 00:55:41.093939 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092611    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 00:55:41.093939 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092614    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 00:55:41.093939 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092616    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 00:55:41.093939 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092621    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 00:55:41.093939 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092625    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 00:55:41.093939 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092628    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 00:55:41.093939 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092631    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 00:55:41.093939 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092634    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 00:55:41.093939 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092637    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 00:55:41.093939 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092639    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 00:55:41.093939 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092642    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 00:55:41.093939 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092645    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 00:55:41.093939 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092648    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 00:55:41.094402 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092650    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 00:55:41.094402 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092653    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 00:55:41.094402 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092656    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 00:55:41.094402 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092658    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 00:55:41.094402 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092661    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 00:55:41.094402 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.092663    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 00:55:41.094402 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093108    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 00:55:41.094402 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093113    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 00:55:41.094402 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093116    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 00:55:41.094402 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093119    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 00:55:41.094402 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093122    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 00:55:41.094402 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093124    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 00:55:41.094402 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093127    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 00:55:41.094402 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093129    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 00:55:41.094402 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093132    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 00:55:41.094402 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093135    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 00:55:41.094402 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093137    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 00:55:41.094402 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093141    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 00:55:41.094402 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093150    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 00:55:41.094402 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093153    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 00:55:41.094901 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093156    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 00:55:41.094901 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093158    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 00:55:41.094901 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093161    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 00:55:41.094901 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093163    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 00:55:41.094901 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093166    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 00:55:41.094901 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093168    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 00:55:41.094901 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093171    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 00:55:41.094901 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093174    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 00:55:41.094901 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093176    2550 feature_gate.go:328] unrecognized feature gate: Example
May 28 00:55:41.094901 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093178    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 00:55:41.094901 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093181    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 00:55:41.094901 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093184    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 00:55:41.094901 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093187    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 00:55:41.094901 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093190    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 00:55:41.094901 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093192    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 00:55:41.094901 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093195    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 00:55:41.094901 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093197    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 00:55:41.094901 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093200    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 00:55:41.094901 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093203    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 00:55:41.094901 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093206    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 00:55:41.095408 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093208    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 00:55:41.095408 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093211    2550 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 00:55:41.095408 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093214    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 00:55:41.095408 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093216    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 00:55:41.095408 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093219    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 00:55:41.095408 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093221    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 00:55:41.095408 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093224    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 00:55:41.095408 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093226    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 00:55:41.095408 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093229    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 00:55:41.095408 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093231    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 00:55:41.095408 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093234    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 00:55:41.095408 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093242    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 00:55:41.095408 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093244    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 00:55:41.095408 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093247    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 00:55:41.095408 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093249    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 00:55:41.095408 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093252    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 00:55:41.095408 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093255    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 00:55:41.095408 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093258    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 00:55:41.095408 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093261    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 00:55:41.095408 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093263    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 00:55:41.095914 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093266    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 00:55:41.095914 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093270    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 00:55:41.095914 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093274    2550 feature_gate.go:328] unrecognized feature gate: Example2
May 28 00:55:41.095914 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093278    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 00:55:41.095914 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093281    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 00:55:41.095914 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093283    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 00:55:41.095914 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093286    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 00:55:41.095914 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093288    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 00:55:41.095914 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093291    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 00:55:41.095914 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093294    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 00:55:41.095914 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093296    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 00:55:41.095914 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093299    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 00:55:41.095914 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093301    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 00:55:41.095914 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093304    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 00:55:41.095914 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093307    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 00:55:41.095914 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093309    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 00:55:41.095914 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093312    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 00:55:41.095914 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093314    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 00:55:41.095914 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093317    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 00:55:41.096374 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093319    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 00:55:41.096374 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093322    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 00:55:41.096374 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093324    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 00:55:41.096374 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093327    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 00:55:41.096374 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093329    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 00:55:41.096374 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093337    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 00:55:41.096374 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093341    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 00:55:41.096374 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093343    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 00:55:41.096374 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093346    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 00:55:41.096374 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093349    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 00:55:41.096374 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093351    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 00:55:41.096374 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093353    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 00:55:41.096374 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.093356    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 00:55:41.096374 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094242    2550 flags.go:64] FLAG: --address="0.0.0.0"
May 28 00:55:41.096374 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094253    2550 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 28 00:55:41.096374 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094262    2550 flags.go:64] FLAG: --anonymous-auth="true"
May 28 00:55:41.096374 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094267    2550 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 28 00:55:41.096374 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094271    2550 flags.go:64] FLAG: --authentication-token-webhook="false"
May 28 00:55:41.096374 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094275    2550 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 28 00:55:41.096374 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094279    2550 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 28 00:55:41.096374 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094284    2550 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 28 00:55:41.096937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094287    2550 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 28 00:55:41.096937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094290    2550 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 28 00:55:41.096937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094293    2550 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 28 00:55:41.096937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094296    2550 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 28 00:55:41.096937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094300    2550 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 28 00:55:41.096937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094303    2550 flags.go:64] FLAG: --cgroup-root=""
May 28 00:55:41.096937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094306    2550 flags.go:64] FLAG: --cgroups-per-qos="true"
May 28 00:55:41.096937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094309    2550 flags.go:64] FLAG: --client-ca-file=""
May 28 00:55:41.096937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094312    2550 flags.go:64] FLAG: --cloud-config=""
May 28 00:55:41.096937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094315    2550 flags.go:64] FLAG: --cloud-provider="external"
May 28 00:55:41.096937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094317    2550 flags.go:64] FLAG: --cluster-dns="[]"
May 28 00:55:41.096937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094325    2550 flags.go:64] FLAG: --cluster-domain=""
May 28 00:55:41.096937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094328    2550 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 28 00:55:41.096937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094331    2550 flags.go:64] FLAG: --config-dir=""
May 28 00:55:41.096937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094334    2550 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 28 00:55:41.096937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094337    2550 flags.go:64] FLAG: --container-log-max-files="5"
May 28 00:55:41.096937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094341    2550 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 28 00:55:41.096937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094350    2550 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 28 00:55:41.096937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094353    2550 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 28 00:55:41.096937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094357    2550 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 28 00:55:41.096937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094360    2550 flags.go:64] FLAG: --contention-profiling="false"
May 28 00:55:41.096937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094362    2550 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 28 00:55:41.096937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094366    2550 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 28 00:55:41.096937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094369    2550 flags.go:64] FLAG: --cpu-manager-policy="none"
May 28 00:55:41.096937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094372    2550 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 28 00:55:41.097634 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094376    2550 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 28 00:55:41.097634 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094379    2550 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 28 00:55:41.097634 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094383    2550 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 28 00:55:41.097634 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094386    2550 flags.go:64] FLAG: --enable-load-reader="false"
May 28 00:55:41.097634 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094389    2550 flags.go:64] FLAG: --enable-server="true"
May 28 00:55:41.097634 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094392    2550 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 28 00:55:41.097634 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094399    2550 flags.go:64] FLAG: --event-burst="100"
May 28 00:55:41.097634 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094402    2550 flags.go:64] FLAG: --event-qps="50"
May 28 00:55:41.097634 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094405    2550 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 28 00:55:41.097634 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094408    2550 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 28 00:55:41.097634 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094425    2550 flags.go:64] FLAG: --eviction-hard=""
May 28 00:55:41.097634 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094429    2550 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 28 00:55:41.097634 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094432    2550 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 28 00:55:41.097634 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094435    2550 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 28 00:55:41.097634 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094438    2550 flags.go:64] FLAG: --eviction-soft=""
May 28 00:55:41.097634 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094441    2550 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 28 00:55:41.097634 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094444    2550 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 28 00:55:41.097634 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094447    2550 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 28 00:55:41.097634 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094450    2550 flags.go:64] FLAG: --experimental-mounter-path=""
May 28 00:55:41.097634 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094453    2550 flags.go:64] FLAG: --fail-cgroupv1="false"
May 28 00:55:41.097634 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094456    2550 flags.go:64] FLAG: --fail-swap-on="true"
May 28 00:55:41.097634 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094459    2550 flags.go:64] FLAG: --feature-gates=""
May 28 00:55:41.097634 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094463    2550 flags.go:64] FLAG: --file-check-frequency="20s"
May 28 00:55:41.097634 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094466    2550 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 28 00:55:41.097634 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094469    2550 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 28 00:55:41.098315 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094479    2550 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 28 00:55:41.098315 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094483    2550 flags.go:64] FLAG: --healthz-port="10248"
May 28 00:55:41.098315 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094486    2550 flags.go:64] FLAG: --help="false"
May 28 00:55:41.098315 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094489    2550 flags.go:64] FLAG: --hostname-override="ip-10-0-131-73.ec2.internal"
May 28 00:55:41.098315 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094492    2550 flags.go:64] FLAG: --housekeeping-interval="10s"
May 28 00:55:41.098315 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094495    2550 flags.go:64] FLAG: --http-check-frequency="20s"
May 28 00:55:41.098315 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094498    2550 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 28 00:55:41.098315 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094502    2550 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 28 00:55:41.098315 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094505    2550 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 28 00:55:41.098315 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094508    2550 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 28 00:55:41.098315 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094511    2550 flags.go:64] FLAG: --image-service-endpoint=""
May 28 00:55:41.098315 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094514    2550 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 28 00:55:41.098315 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094517    2550 flags.go:64] FLAG: --kube-api-burst="100"
May 28 00:55:41.098315 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094520    2550 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 28 00:55:41.098315 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094523    2550 flags.go:64] FLAG: --kube-api-qps="50"
May 28 00:55:41.098315 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094526    2550 flags.go:64] FLAG: --kube-reserved=""
May 28 00:55:41.098315 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094529    2550 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 28 00:55:41.098315 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094532    2550 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 28 00:55:41.098315 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094535    2550 flags.go:64] FLAG: --kubelet-cgroups=""
May 28 00:55:41.098315 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094537    2550 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 28 00:55:41.098315 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094540    2550 flags.go:64] FLAG: --lock-file=""
May 28 00:55:41.098315 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094543    2550 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 28 00:55:41.098315 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094546    2550 flags.go:64] FLAG: --log-flush-frequency="5s"
May 28 00:55:41.098315 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094549    2550 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 28 00:55:41.098904 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094554    2550 flags.go:64] FLAG: --log-json-split-stream="false"
May 28 00:55:41.098904 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094557    2550 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 28 00:55:41.098904 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094559    2550 flags.go:64] FLAG: --log-text-split-stream="false"
May 28 00:55:41.098904 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094562    2550 flags.go:64] FLAG: --logging-format="text"
May 28 00:55:41.098904 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094565    2550 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 28 00:55:41.098904 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094568    2550 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 28 00:55:41.098904 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094571    2550 flags.go:64] FLAG: --manifest-url=""
May 28 00:55:41.098904 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094574    2550 flags.go:64] FLAG: --manifest-url-header=""
May 28 00:55:41.098904 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094578    2550 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 28 00:55:41.098904 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094588    2550 flags.go:64] FLAG: --max-open-files="1000000"
May 28 00:55:41.098904 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094592    2550 flags.go:64] FLAG: --max-pods="110"
May 28 00:55:41.098904 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094595    2550 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 28 00:55:41.098904 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094598    2550 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 28 00:55:41.098904 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094601    2550 flags.go:64] FLAG: --memory-manager-policy="None"
May 28 00:55:41.098904 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094604    2550 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 28 00:55:41.098904 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094607    2550 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 28 00:55:41.098904 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094610    2550 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 28 00:55:41.098904 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094613    2550 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 28 00:55:41.098904 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094621    2550 flags.go:64] FLAG: --node-status-max-images="50"
May 28 00:55:41.098904 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094624    2550 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 28 00:55:41.098904 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094627    2550 flags.go:64] FLAG: --oom-score-adj="-999"
May 28 00:55:41.098904 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094630    2550 flags.go:64] FLAG: --pod-cidr=""
May 28 00:55:41.098904 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094633    2550 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc"
May 28 00:55:41.099485 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094639    2550 flags.go:64] FLAG: --pod-manifest-path=""
May 28 00:55:41.099485 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094642    2550 flags.go:64] FLAG: --pod-max-pids="-1"
May 28 00:55:41.099485 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094646    2550 flags.go:64] FLAG: --pods-per-core="0"
May 28 00:55:41.099485 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094649    2550 flags.go:64] FLAG: --port="10250"
May 28 00:55:41.099485 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094651    2550 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 28 00:55:41.099485 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094654    2550 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06fe8a8fc085de091"
May 28 00:55:41.099485 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094658    2550 flags.go:64] FLAG: --qos-reserved=""
May 28 00:55:41.099485 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094661    2550 flags.go:64] FLAG: --read-only-port="10255"
May 28 00:55:41.099485 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094664    2550 flags.go:64] FLAG: --register-node="true"
May 28 00:55:41.099485 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094666    2550 flags.go:64] FLAG: --register-schedulable="true"
May 28 00:55:41.099485 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094670    2550 flags.go:64] FLAG: --register-with-taints=""
May 28 00:55:41.099485 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094673    2550 flags.go:64] FLAG: --registry-burst="10"
May 28 00:55:41.099485 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094676    2550 flags.go:64] FLAG: --registry-qps="5"
May 28 00:55:41.099485 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094679    2550 flags.go:64] FLAG: --reserved-cpus=""
May 28 00:55:41.099485 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094682    2550 flags.go:64] FLAG: --reserved-memory=""
May 28 00:55:41.099485 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094685    2550 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 28 00:55:41.099485 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094689    2550 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 28 00:55:41.099485 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094692    2550 flags.go:64] FLAG: --rotate-certificates="false"
May 28 00:55:41.099485 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094695    2550 flags.go:64] FLAG: --rotate-server-certificates="false"
May 28 00:55:41.099485 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094704    2550 flags.go:64] FLAG: --runonce="false"
May 28 00:55:41.099485 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094707    2550 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 28 00:55:41.099485 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094710    2550 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 28 00:55:41.099485 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094713    2550 flags.go:64] FLAG: --seccomp-default="false"
May 28 00:55:41.099485 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094716    2550 flags.go:64] FLAG: --serialize-image-pulls="true"
May 28 00:55:41.099485 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094719    2550 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 28 00:55:41.099485 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094722    2550 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 28 00:55:41.100126 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094728    2550 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 28 00:55:41.100126 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094731    2550 flags.go:64] FLAG: --storage-driver-password="root"
May 28 00:55:41.100126 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094734    2550 flags.go:64] FLAG: --storage-driver-secure="false"
May 28 00:55:41.100126 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094737    2550 flags.go:64] FLAG: --storage-driver-table="stats"
May 28 00:55:41.100126 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094740    2550 flags.go:64] FLAG: --storage-driver-user="root"
May 28 00:55:41.100126 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094743    2550 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 28 00:55:41.100126 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094746    2550 flags.go:64] FLAG: --sync-frequency="1m0s"
May 28 00:55:41.100126 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094749    2550 flags.go:64] FLAG: --system-cgroups=""
May 28 00:55:41.100126 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094752    2550 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 28 00:55:41.100126 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094757    2550 flags.go:64] FLAG: --system-reserved-cgroup=""
May 28 00:55:41.100126 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094760    2550 flags.go:64] FLAG: --tls-cert-file=""
May 28 00:55:41.100126 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094763    2550 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 28 00:55:41.100126 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094770    2550 flags.go:64] FLAG: --tls-min-version=""
May 28 00:55:41.100126 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094773    2550 flags.go:64] FLAG: --tls-private-key-file=""
May 28 00:55:41.100126 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094775    2550 flags.go:64] FLAG: --topology-manager-policy="none"
May 28 00:55:41.100126 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094778    2550 flags.go:64] FLAG: --topology-manager-policy-options=""
May 28 00:55:41.100126 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094781    2550 flags.go:64] FLAG: --topology-manager-scope="container"
May 28 00:55:41.100126 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094784    2550 flags.go:64] FLAG: --v="2"
May 28 00:55:41.100126 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094788    2550 flags.go:64] FLAG: --version="false"
May 28 00:55:41.100126 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094792    2550 flags.go:64] FLAG: --vmodule=""
May 28 00:55:41.100126 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094796    2550 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 28 00:55:41.100126 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.094800    2550 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 28 00:55:41.100126 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094921    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 00:55:41.100126 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094925    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 00:55:41.100743 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094929    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 00:55:41.100743 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094932    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 00:55:41.100743 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094935    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 00:55:41.100743 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094939    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 00:55:41.100743 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094943    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 00:55:41.100743 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094946    2550 feature_gate.go:328] unrecognized feature gate: Example2
May 28 00:55:41.100743 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094950    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 00:55:41.100743 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094952    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 00:55:41.100743 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094956    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 00:55:41.100743 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094958    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 00:55:41.100743 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094961    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 00:55:41.100743 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094964    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 00:55:41.100743 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094966    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 00:55:41.100743 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094969    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 00:55:41.100743 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094971    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 00:55:41.100743 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094974    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 00:55:41.100743 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094976    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 00:55:41.100743 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094979    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 00:55:41.100743 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094982    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 00:55:41.100743 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094984    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 00:55:41.101254 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094987    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 00:55:41.101254 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094989    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 00:55:41.101254 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094992    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 00:55:41.101254 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094994    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 00:55:41.101254 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094997    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 00:55:41.101254 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.094999    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 00:55:41.101254 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095002    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 00:55:41.101254 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095004    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 00:55:41.101254 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095007    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 00:55:41.101254 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095011    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 00:55:41.101254 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095014    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 00:55:41.101254 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095017    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 00:55:41.101254 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095021    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 00:55:41.101254 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095025    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 00:55:41.101254 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095028    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 00:55:41.101254 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095032    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 00:55:41.101254 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095034    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 00:55:41.101254 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095037    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 00:55:41.101254 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095040    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 00:55:41.101828 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095042    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 00:55:41.101828 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095045    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 00:55:41.101828 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095047    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 00:55:41.101828 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095050    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 00:55:41.101828 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095053    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 00:55:41.101828 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095055    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 00:55:41.101828 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095058    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 00:55:41.101828 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095061    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 00:55:41.101828 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095063    2550 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 00:55:41.101828 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095066    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 00:55:41.101828 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095069    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 00:55:41.101828 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095071    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 00:55:41.101828 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095074    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 00:55:41.101828 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095076    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 00:55:41.101828 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095079    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 00:55:41.101828 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095081    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 00:55:41.101828 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095084    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 00:55:41.101828 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095086    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 00:55:41.101828 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095089    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 00:55:41.101828 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095091    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 00:55:41.102318 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095094    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 00:55:41.102318 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095097    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 00:55:41.102318 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095100    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 00:55:41.102318 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095102    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 00:55:41.102318 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095105    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 00:55:41.102318 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095109    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 00:55:41.102318 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095112    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 00:55:41.102318 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095115    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 00:55:41.102318 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095117    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 00:55:41.102318 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095120    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 00:55:41.102318 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095123    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 00:55:41.102318 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095125    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 00:55:41.102318 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095128    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 00:55:41.102318 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095131    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 00:55:41.102318 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095133    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 00:55:41.102318 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095135    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 00:55:41.102318 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095138    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 00:55:41.102318 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095140    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 00:55:41.102318 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095143    2550 feature_gate.go:328] unrecognized feature gate: Example
May 28 00:55:41.102318 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095145    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 00:55:41.102826 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095148    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 00:55:41.102826 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095150    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 00:55:41.102826 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095153    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 00:55:41.102826 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095155    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 00:55:41.102826 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.095158    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 00:55:41.102826 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.095163    2550 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 00:55:41.102826 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.102063    2550 server.go:530] "Kubelet version" kubeletVersion="v1.33.11"
May 28 00:55:41.102826 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.102086    2550 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 28 00:55:41.102826 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102134    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 00:55:41.102826 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102140    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 00:55:41.102826 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102143    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 00:55:41.102826 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102146    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 00:55:41.102826 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102149    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 00:55:41.102826 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102152    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 00:55:41.102826 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102155    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 00:55:41.103194 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102158    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 00:55:41.103194 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102161    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 00:55:41.103194 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102163    2550 feature_gate.go:328] unrecognized feature gate: Example
May 28 00:55:41.103194 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102166    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 00:55:41.103194 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102169    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 00:55:41.103194 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102172    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 00:55:41.103194 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102174    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 00:55:41.103194 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102177    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 00:55:41.103194 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102180    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 00:55:41.103194 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102184    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 00:55:41.103194 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102188    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 00:55:41.103194 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102192    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 00:55:41.103194 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102195    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 00:55:41.103194 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102198    2550 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 00:55:41.103194 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102200    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 00:55:41.103194 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102203    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 00:55:41.103194 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102206    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 00:55:41.103194 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102208    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 00:55:41.103194 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102212    2550 feature_gate.go:328] unrecognized feature gate: Example2
May 28 00:55:41.103194 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102214    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 00:55:41.103729 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102217    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 00:55:41.103729 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102220    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 00:55:41.103729 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102223    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 00:55:41.103729 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102226    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 00:55:41.103729 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102236    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 00:55:41.103729 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102239    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 00:55:41.103729 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102242    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 00:55:41.103729 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102244    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 00:55:41.103729 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102247    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 00:55:41.103729 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102250    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 00:55:41.103729 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102252    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 00:55:41.103729 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102255    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 00:55:41.103729 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102258    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 00:55:41.103729 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102260    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 00:55:41.103729 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102263    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 00:55:41.103729 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102266    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 00:55:41.103729 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102268    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 00:55:41.103729 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102271    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 00:55:41.103729 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102273    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 00:55:41.103729 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102276    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 00:55:41.104219 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102279    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 00:55:41.104219 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102281    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 00:55:41.104219 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102284    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 00:55:41.104219 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102287    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 00:55:41.104219 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102289    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 00:55:41.104219 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102292    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 00:55:41.104219 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102295    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 00:55:41.104219 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102297    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 00:55:41.104219 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102300    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 00:55:41.104219 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102304    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 00:55:41.104219 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102308    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 00:55:41.104219 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102311    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 00:55:41.104219 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102314    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 00:55:41.104219 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102318    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 00:55:41.104219 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102321    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 00:55:41.104219 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102323    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 00:55:41.104219 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102327    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 00:55:41.104219 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102337    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 00:55:41.104219 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102340    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 00:55:41.104716 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102342    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 00:55:41.104716 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102345    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 00:55:41.104716 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102348    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 00:55:41.104716 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102350    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 00:55:41.104716 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102353    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 00:55:41.104716 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102355    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 00:55:41.104716 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102358    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 00:55:41.104716 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102360    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 00:55:41.104716 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102363    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 00:55:41.104716 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102365    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 00:55:41.104716 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102368    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 00:55:41.104716 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102370    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 00:55:41.104716 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102373    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 00:55:41.104716 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102375    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 00:55:41.104716 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102378    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 00:55:41.104716 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102381    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 00:55:41.104716 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102383    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 00:55:41.104716 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102386    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 00:55:41.104716 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102389    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 00:55:41.104716 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102391    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 00:55:41.105201 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.102396    2550 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 00:55:41.105201 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102566    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 00:55:41.105201 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102572    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 00:55:41.105201 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102575    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 00:55:41.105201 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102579    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 00:55:41.105201 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102581    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 00:55:41.105201 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102584    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 00:55:41.105201 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102587    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 00:55:41.105201 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102590    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 00:55:41.105201 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102593    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 00:55:41.105201 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102596    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 00:55:41.105201 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102605    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 00:55:41.105201 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102608    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 00:55:41.105201 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102611    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 00:55:41.105201 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102615    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 00:55:41.105668 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102620    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 00:55:41.105668 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102623    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 00:55:41.105668 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102626    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 00:55:41.105668 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102629    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 00:55:41.105668 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102631    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 00:55:41.105668 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102634    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 00:55:41.105668 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102636    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 00:55:41.105668 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102646    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 00:55:41.105668 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102649    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 00:55:41.105668 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102651    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 00:55:41.105668 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102654    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 00:55:41.105668 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102657    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 00:55:41.105668 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102659    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 00:55:41.105668 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102662    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 00:55:41.105668 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102664    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 00:55:41.105668 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102667    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 00:55:41.105668 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102669    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 00:55:41.105668 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102672    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 00:55:41.105668 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102674    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 00:55:41.105668 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102676    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 00:55:41.106149 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102679    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 00:55:41.106149 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102682    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 00:55:41.106149 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102684    2550 feature_gate.go:328] unrecognized feature gate: Example
May 28 00:55:41.106149 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102687    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 00:55:41.106149 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102689    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 00:55:41.106149 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102692    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 00:55:41.106149 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102695    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 00:55:41.106149 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102697    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 00:55:41.106149 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102700    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 00:55:41.106149 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102709    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 00:55:41.106149 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102711    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 00:55:41.106149 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102714    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 00:55:41.106149 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102717    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 00:55:41.106149 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102719    2550 feature_gate.go:328] unrecognized feature gate: Example2
May 28 00:55:41.106149 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102722    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 00:55:41.106149 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102724    2550 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 00:55:41.106149 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102727    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 00:55:41.106149 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102729    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 00:55:41.106149 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102732    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 00:55:41.106149 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102734    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 00:55:41.106646 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102737    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 00:55:41.106646 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102739    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 00:55:41.106646 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102742    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 00:55:41.106646 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102744    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 00:55:41.106646 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102747    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 00:55:41.106646 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102750    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 00:55:41.106646 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102752    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 00:55:41.106646 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102755    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 00:55:41.106646 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102757    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 00:55:41.106646 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102760    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 00:55:41.106646 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102762    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 00:55:41.106646 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102765    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 00:55:41.106646 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102767    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 00:55:41.106646 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102771    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 00:55:41.106646 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102775    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 00:55:41.106646 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102778    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 00:55:41.106646 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102780    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 00:55:41.106646 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102783    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 00:55:41.106646 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102786    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 00:55:41.107110 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102790    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 00:55:41.107110 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102793    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 00:55:41.107110 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102796    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 00:55:41.107110 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102804    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 00:55:41.107110 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102808    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 00:55:41.107110 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102810    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 00:55:41.107110 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102813    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 00:55:41.107110 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102816    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 00:55:41.107110 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102819    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 00:55:41.107110 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102821    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 00:55:41.107110 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102824    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 00:55:41.107110 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102827    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 00:55:41.107110 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:41.102829    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 00:55:41.107110 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.102834    2550 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 00:55:41.107110 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.103740    2550 server.go:962] "Client rotation is on, will bootstrap in background"
May 28 00:55:41.107487 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.107159    2550 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 28 00:55:41.108286 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.108273    2550 server.go:1019] "Starting client certificate rotation"
May 28 00:55:41.108388 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.108372    2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 00:55:41.108435 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.108426    2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 00:55:41.136815 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.136794    2550 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 00:55:41.143362 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.143316    2550 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 00:55:41.156691 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.156667    2550 log.go:25] "Validated CRI v1 runtime API"
May 28 00:55:41.163242 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.163220    2550 log.go:25] "Validated CRI v1 image API"
May 28 00:55:41.164716 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.164693    2550 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 28 00:55:41.168384 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.168359    2550 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a3643886-feef-47e2-95c2-88817fd430fe:/dev/nvme0n1p4 e14e846f-fef2-4ae7-a06e-314dab68b859:/dev/nvme0n1p3]
May 28 00:55:41.168475 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.168383    2550 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 28 00:55:41.170246 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.170228    2550 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 00:55:41.173150 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.173038    2550 manager.go:217] Machine: {Timestamp:2026-05-28 00:55:41.172544056 +0000 UTC m=+0.453501140 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098826 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d0dd95896bb2a6d03e7eacfedf902 SystemUUID:ec2d0dd9-5896-bb2a-6d03-e7eacfedf902 BootID:5a35359f-e76b-42de-8e50-1e7945701491 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:46:39:7c:4b:1d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:46:39:7c:4b:1d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:96:83:8e:7d:dd:c7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 28 00:55:41.174015 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.174004    2550 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 28 00:55:41.174129 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.174117    2550 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 28 00:55:41.177129 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.177100    2550 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 28 00:55:41.177271 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.177132    2550 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-73.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 28 00:55:41.177318 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.177284    2550 topology_manager.go:138] "Creating topology manager with none policy"
May 28 00:55:41.177318 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.177292    2550 container_manager_linux.go:306] "Creating device plugin manager"
May 28 00:55:41.177318 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.177300    2550 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 00:55:41.178272 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.178261    2550 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 00:55:41.180191 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.180181    2550 state_mem.go:36] "Initialized new in-memory state store"
May 28 00:55:41.180307 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.180297    2550 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 28 00:55:41.183118 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.183109    2550 kubelet.go:491] "Attempting to sync node with API server"
May 28 00:55:41.183159 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.183129    2550 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 28 00:55:41.183159 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.183141    2550 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 28 00:55:41.183159 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.183150    2550 kubelet.go:397] "Adding apiserver pod source"
May 28 00:55:41.183270 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.183164    2550 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 28 00:55:41.184466 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.184454    2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 00:55:41.184511 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.184473    2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 00:55:41.187482 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.187462    2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-f2zqp"
May 28 00:55:41.187738 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.187717    2550 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 28 00:55:41.193119 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.193101    2550 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 28 00:55:41.193487 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:41.193452    2550 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 28 00:55:41.193618 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:41.193597    2550 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-73.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 28 00:55:41.194845 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.194829    2550 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 28 00:55:41.194928 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.194852    2550 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 28 00:55:41.194928 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.194861    2550 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 28 00:55:41.194928 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.194866    2550 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 28 00:55:41.194928 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.194877    2550 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 28 00:55:41.194928 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.194884    2550 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 28 00:55:41.194928 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.194890    2550 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 28 00:55:41.194928 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.194896    2550 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 28 00:55:41.194928 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.194903    2550 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 28 00:55:41.194928 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.194909    2550 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 28 00:55:41.194928 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.194927    2550 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 28 00:55:41.195356 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.195231    2550 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 28 00:55:41.195705 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.195688    2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-f2zqp"
May 28 00:55:41.196342 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.196325    2550 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 28 00:55:41.196342 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.196342    2550 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 28 00:55:41.197719 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.197701    2550 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-73.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 28 00:55:41.200452 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.200439    2550 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 28 00:55:41.200502 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.200478    2550 server.go:1295] "Started kubelet"
May 28 00:55:41.200573 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.200537    2550 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 28 00:55:41.200618 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.200558    2550 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 28 00:55:41.200666 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.200620    2550 server_v1.go:47] "podresources" method="list" useActivePods=true
May 28 00:55:41.201173 ip-10-0-131-73 systemd[1]: Started Kubernetes Kubelet.
May 28 00:55:41.202747 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.202731    2550 server.go:317] "Adding debug handlers to kubelet server"
May 28 00:55:41.202815 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.202756    2550 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 28 00:55:41.209665 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.209649    2550 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 28 00:55:41.209765 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.209664    2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 28 00:55:41.209765 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.209738    2550 volume_manager.go:295] "The desired_state_of_world populator starts"
May 28 00:55:41.209765 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.209751    2550 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 28 00:55:41.210032 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.210019    2550 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 28 00:55:41.210108 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:41.210091    2550 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 28 00:55:41.210366 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.210350    2550 reconstruct.go:97] "Volume reconstruction finished"
May 28 00:55:41.210366 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.210364    2550 reconciler.go:26] "Reconciler: start to sync state"
May 28 00:55:41.210654 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.210481    2550 factory.go:153] Registering CRI-O factory
May 28 00:55:41.210749 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:41.210682    2550 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-73.ec2.internal\" not found"
May 28 00:55:41.210749 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.210739    2550 factory.go:223] Registration of the crio container factory successfully
May 28 00:55:41.210876 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.210849    2550 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 28 00:55:41.210876 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.210861    2550 factory.go:55] Registering systemd factory
May 28 00:55:41.210876 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.210871    2550 factory.go:223] Registration of the systemd container factory successfully
May 28 00:55:41.211224 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.210890    2550 factory.go:103] Registering Raw factory
May 28 00:55:41.211224 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.210904    2550 manager.go:1196] Started watching for new ooms in manager
May 28 00:55:41.211357 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.211340    2550 manager.go:319] Starting recovery of all containers
May 28 00:55:41.215623 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.215599    2550 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 00:55:41.222318 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:41.222291    2550 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-131-73.ec2.internal\" not found" node="ip-10-0-131-73.ec2.internal"
May 28 00:55:41.224244 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.224127    2550 manager.go:324] Recovery completed
May 28 00:55:41.229139 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.229123    2550 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 00:55:41.230999 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.230983    2550 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-73.ec2.internal" event="NodeHasSufficientMemory"
May 28 00:55:41.231088 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.231016    2550 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-73.ec2.internal" event="NodeHasNoDiskPressure"
May 28 00:55:41.231088 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.231032    2550 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-73.ec2.internal" event="NodeHasSufficientPID"
May 28 00:55:41.231615 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.231598    2550 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 28 00:55:41.231615 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.231614    2550 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 28 00:55:41.231704 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.231631    2550 state_mem.go:36] "Initialized new in-memory state store"
May 28 00:55:41.235572 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.235560    2550 policy_none.go:49] "None policy: Start"
May 28 00:55:41.235611 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.235577    2550 memory_manager.go:186] "Starting memorymanager" policy="None"
May 28 00:55:41.235611 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.235587    2550 state_mem.go:35] "Initializing new in-memory state store"
May 28 00:55:41.279080 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.279062    2550 manager.go:341] "Starting Device Plugin manager"
May 28 00:55:41.299520 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:41.279098    2550 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 28 00:55:41.299520 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.279108    2550 server.go:85] "Starting device plugin registration server"
May 28 00:55:41.299520 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.279328    2550 eviction_manager.go:189] "Eviction manager: starting control loop"
May 28 00:55:41.299520 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.279337    2550 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 28 00:55:41.299520 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.279451    2550 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 28 00:55:41.299520 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.279534    2550 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 28 00:55:41.299520 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.279545    2550 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 28 00:55:41.299520 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:41.280050    2550 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 28 00:55:41.299520 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:41.280102    2550 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-73.ec2.internal\" not found"
May 28 00:55:41.317703 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.317665    2550 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 28 00:55:41.318867 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.318851    2550 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 28 00:55:41.318931 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.318881    2550 status_manager.go:230] "Starting to sync pod status with apiserver"
May 28 00:55:41.318931 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.318900    2550 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 28 00:55:41.318931 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.318906    2550 kubelet.go:2451] "Starting kubelet main sync loop"
May 28 00:55:41.319064 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:41.318942    2550 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 28 00:55:41.323043 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.323027    2550 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 00:55:41.380148 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.380082    2550 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 00:55:41.383600 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.383582    2550 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-73.ec2.internal" event="NodeHasSufficientMemory"
May 28 00:55:41.383687 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.383617    2550 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-73.ec2.internal" event="NodeHasNoDiskPressure"
May 28 00:55:41.383687 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.383630    2550 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-73.ec2.internal" event="NodeHasSufficientPID"
May 28 00:55:41.383687 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.383653    2550 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-73.ec2.internal"
May 28 00:55:41.391947 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.391922    2550 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-73.ec2.internal"
May 28 00:55:41.391998 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:41.391956    2550 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-73.ec2.internal\": node \"ip-10-0-131-73.ec2.internal\" not found"
May 28 00:55:41.417838 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:41.417810    2550 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-73.ec2.internal\" not found"
May 28 00:55:41.419872 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.419835    2550 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-73.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-73.ec2.internal"]
May 28 00:55:41.419977 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.419916    2550 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 00:55:41.421474 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.421459    2550 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-73.ec2.internal" event="NodeHasSufficientMemory"
May 28 00:55:41.421535 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.421487    2550 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-73.ec2.internal" event="NodeHasNoDiskPressure"
May 28 00:55:41.421535 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.421497    2550 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-73.ec2.internal" event="NodeHasSufficientPID"
May 28 00:55:41.422857 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.422843    2550 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 00:55:41.423035 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.423019    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-73.ec2.internal"
May 28 00:55:41.423097 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.423056    2550 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 00:55:41.423937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.423909    2550 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-73.ec2.internal" event="NodeHasSufficientMemory"
May 28 00:55:41.423937 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.423935    2550 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-73.ec2.internal" event="NodeHasNoDiskPressure"
May 28 00:55:41.424089 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.423943    2550 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-73.ec2.internal" event="NodeHasSufficientMemory"
May 28 00:55:41.424089 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.423968    2550 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-73.ec2.internal" event="NodeHasNoDiskPressure"
May 28 00:55:41.424089 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.423983    2550 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-73.ec2.internal" event="NodeHasSufficientPID"
May 28 00:55:41.424089 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.423948    2550 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-73.ec2.internal" event="NodeHasSufficientPID"
May 28 00:55:41.425330 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.425313    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-73.ec2.internal"
May 28 00:55:41.425441 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.425344    2550 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 00:55:41.426038 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.426022    2550 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-73.ec2.internal" event="NodeHasSufficientMemory"
May 28 00:55:41.426132 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.426046    2550 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-73.ec2.internal" event="NodeHasNoDiskPressure"
May 28 00:55:41.426132 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.426060    2550 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-73.ec2.internal" event="NodeHasSufficientPID"
May 28 00:55:41.440970 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:41.440949    2550 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-73.ec2.internal\" not found" node="ip-10-0-131-73.ec2.internal"
May 28 00:55:41.444705 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:41.444686    2550 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-73.ec2.internal\" not found" node="ip-10-0-131-73.ec2.internal"
May 28 00:55:41.513256 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.513230    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4c51bc3462f829dd646da56091097a15-config\") pod \"kube-apiserver-proxy-ip-10-0-131-73.ec2.internal\" (UID: \"4c51bc3462f829dd646da56091097a15\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-73.ec2.internal"
May 28 00:55:41.513431 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.513261    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/00e8ca491df3bef53c3295016d026193-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-73.ec2.internal\" (UID: \"00e8ca491df3bef53c3295016d026193\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-73.ec2.internal"
May 28 00:55:41.513431 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.513280    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00e8ca491df3bef53c3295016d026193-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-73.ec2.internal\" (UID: \"00e8ca491df3bef53c3295016d026193\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-73.ec2.internal"
May 28 00:55:41.518371 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:41.518328    2550 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-73.ec2.internal\" not found"
May 28 00:55:41.613745 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.613708    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4c51bc3462f829dd646da56091097a15-config\") pod \"kube-apiserver-proxy-ip-10-0-131-73.ec2.internal\" (UID: \"4c51bc3462f829dd646da56091097a15\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-73.ec2.internal"
May 28 00:55:41.613828 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.613750    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/00e8ca491df3bef53c3295016d026193-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-73.ec2.internal\" (UID: \"00e8ca491df3bef53c3295016d026193\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-73.ec2.internal"
May 28 00:55:41.613828 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.613767    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00e8ca491df3bef53c3295016d026193-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-73.ec2.internal\" (UID: \"00e8ca491df3bef53c3295016d026193\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-73.ec2.internal"
May 28 00:55:41.613828 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.613807    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4c51bc3462f829dd646da56091097a15-config\") pod \"kube-apiserver-proxy-ip-10-0-131-73.ec2.internal\" (UID: \"4c51bc3462f829dd646da56091097a15\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-73.ec2.internal"
May 28 00:55:41.613828 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.613823    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00e8ca491df3bef53c3295016d026193-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-73.ec2.internal\" (UID: \"00e8ca491df3bef53c3295016d026193\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-73.ec2.internal"
May 28 00:55:41.613950 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.613824    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/00e8ca491df3bef53c3295016d026193-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-73.ec2.internal\" (UID: \"00e8ca491df3bef53c3295016d026193\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-73.ec2.internal"
May 28 00:55:41.618817 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:41.618801    2550 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-73.ec2.internal\" not found"
May 28 00:55:41.719659 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:41.719584    2550 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-73.ec2.internal\" not found"
May 28 00:55:41.743775 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.743750    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-73.ec2.internal"
May 28 00:55:41.747575 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:41.747553    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-73.ec2.internal"
May 28 00:55:41.819935 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:41.819887    2550 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-73.ec2.internal\" not found"
May 28 00:55:41.920459 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:41.920431    2550 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-73.ec2.internal\" not found"
May 28 00:55:42.020983 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:42.020907    2550 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-73.ec2.internal\" not found"
May 28 00:55:42.109135 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:42.109105    2550 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 28 00:55:42.109759 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:42.109265    2550 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 28 00:55:42.109759 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:42.109289    2550 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 28 00:55:42.121482 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:42.121458    2550 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-73.ec2.internal\" not found"
May 28 00:55:42.198121 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:42.198076    2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-27 00:50:41 +0000 UTC" deadline="2028-02-08 23:16:49.008769322 +0000 UTC"
May 28 00:55:42.198121 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:42.198116    2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14926h21m6.810655965s"
May 28 00:55:42.210673 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:42.210651    2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 28 00:55:42.221798 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:42.221776    2550 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-73.ec2.internal\" not found"
May 28 00:55:42.221903 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:42.221818    2550 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 00:55:42.247950 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:42.247923    2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-sc8zl"
May 28 00:55:42.257114 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:42.257098    2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-sc8zl"
May 28 00:55:42.264748 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:42.264712    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00e8ca491df3bef53c3295016d026193.slice/crio-4adf37a6796ff16713e2934fd6b84af8cea15eaad116c6f324ce79e4417f17b4 WatchSource:0}: Error finding container 4adf37a6796ff16713e2934fd6b84af8cea15eaad116c6f324ce79e4417f17b4: Status 404 returned error can't find the container with id 4adf37a6796ff16713e2934fd6b84af8cea15eaad116c6f324ce79e4417f17b4
May 28 00:55:42.265005 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:42.264988    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c51bc3462f829dd646da56091097a15.slice/crio-46b6fb5b0a8fcaa918bb0677e444a4e25e9a2fc401c2e2b97c6c389db3a00305 WatchSource:0}: Error finding container 46b6fb5b0a8fcaa918bb0677e444a4e25e9a2fc401c2e2b97c6c389db3a00305: Status 404 returned error can't find the container with id 46b6fb5b0a8fcaa918bb0677e444a4e25e9a2fc401c2e2b97c6c389db3a00305
May 28 00:55:42.268082 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:42.268067    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 00:55:42.321525 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:42.321478    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-73.ec2.internal" event={"ID":"00e8ca491df3bef53c3295016d026193","Type":"ContainerStarted","Data":"4adf37a6796ff16713e2934fd6b84af8cea15eaad116c6f324ce79e4417f17b4"}
May 28 00:55:42.321853 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:42.321821    2550 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-73.ec2.internal\" not found"
May 28 00:55:42.322507 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:42.322487    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-73.ec2.internal" event={"ID":"4c51bc3462f829dd646da56091097a15","Type":"ContainerStarted","Data":"46b6fb5b0a8fcaa918bb0677e444a4e25e9a2fc401c2e2b97c6c389db3a00305"}
May 28 00:55:42.421923 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:42.421892    2550 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-131-73.ec2.internal\" not found"
May 28 00:55:42.506870 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:42.506838    2550 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 00:55:42.508583 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:42.508567    2550 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 00:55:42.509988 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:42.509974    2550 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-73.ec2.internal"
May 28 00:55:42.522370 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:42.522320    2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 00:55:42.523448 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:42.523406    2550 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-73.ec2.internal"
May 28 00:55:42.533476 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:42.533457    2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 00:55:43.059553 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.059510    2550 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 00:55:43.184632 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.184598    2550 apiserver.go:52] "Watching apiserver"
May 28 00:55:43.191444 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.191401    2550 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 28 00:55:43.192886 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.192855    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-9z98m","kube-system/kube-apiserver-proxy-ip-10-0-131-73.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg","openshift-dns/node-resolver-m47th","openshift-image-registry/node-ca-qxmtt","openshift-multus/network-metrics-daemon-f8h8b","openshift-network-operator/iptables-alerter-nj698","kube-system/konnectivity-agent-lhxvp","openshift-cluster-node-tuning-operator/tuned-z8xgs","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-73.ec2.internal","openshift-multus/multus-additional-cni-plugins-qn2qw","openshift-multus/multus-vxdn6","openshift-network-diagnostics/network-check-target-l9q4k"]
May 28 00:55:43.195811 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.195785    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nj698"
May 28 00:55:43.198156 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.197826    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg"
May 28 00:55:43.198265 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.198168    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 28 00:55:43.198265 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.198185    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 28 00:55:43.198375 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.198291    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wl8dp\""
May 28 00:55:43.198375 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.198344    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 28 00:55:43.199964 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.199942    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 28 00:55:43.200067 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.199989    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sv7tf\""
May 28 00:55:43.200329 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.200311    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 28 00:55:43.200401 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.200350    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 28 00:55:43.202171 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.202150    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-m47th"
May 28 00:55:43.204371 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.204044    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cj79s\""
May 28 00:55:43.204371 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.204114    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 28 00:55:43.204371 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.204225    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 28 00:55:43.204371 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.204331    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qxmtt"
May 28 00:55:43.206031 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.206013    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 28 00:55:43.206126 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.206099    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 28 00:55:43.206189 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.206135    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fwgv9\""
May 28 00:55:43.206298 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.206276    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 28 00:55:43.206933 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.206890    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.208627 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.208609    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 28 00:55:43.208704 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.208657    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 28 00:55:43.208919 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.208902    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 28 00:55:43.208994 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.208955    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 28 00:55:43.209049 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.209005    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qk7rv\""
May 28 00:55:43.209049 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.209032    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 28 00:55:43.209284 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.209258    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 28 00:55:43.209370 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.209303    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:55:43.209451 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:43.209409    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8h8b" podUID="3d04c1e7-069a-413e-a9f6-d87a8095844d"
May 28 00:55:43.211581 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.211563    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lhxvp"
May 28 00:55:43.211673 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.211583    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.213543 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.213522    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xlbqv\""
May 28 00:55:43.213678 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.213557    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 28 00:55:43.213893 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.213769    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 28 00:55:43.213893 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.213802    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 28 00:55:43.213893 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.213837    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 28 00:55:43.213893 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.213851    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-w6s4d\""
May 28 00:55:43.218575 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.218554    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qn2qw"
May 28 00:55:43.218673 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.218622    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.220627 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.220602    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 28 00:55:43.220627 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.220617    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 28 00:55:43.220769 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.220681    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 28 00:55:43.220886 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.220867    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6fp2b\""
May 28 00:55:43.220969 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.220923    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 28 00:55:43.220969 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.220932    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 28 00:55:43.221065 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.220989    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:55:43.221065 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:43.221036    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9q4k" podUID="6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7"
May 28 00:55:43.221150 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.221131    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qd5xl\""
May 28 00:55:43.221197 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.221177    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 28 00:55:43.221273 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.221254    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm4d7\" (UniqueName: \"kubernetes.io/projected/a4ecd41b-62cb-4336-9b2b-d082dd27c847-kube-api-access-gm4d7\") pod \"iptables-alerter-nj698\" (UID: \"a4ecd41b-62cb-4336-9b2b-d082dd27c847\") " pod="openshift-network-operator/iptables-alerter-nj698"
May 28 00:55:43.221348 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.221292    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1f856cae-503d-46e6-83f2-ef3127046b5c-konnectivity-ca\") pod \"konnectivity-agent-lhxvp\" (UID: \"1f856cae-503d-46e6-83f2-ef3127046b5c\") " pod="kube-system/konnectivity-agent-lhxvp"
May 28 00:55:43.221348 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.221320    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba57d944-f539-4167-be8f-dd6a88026256-tmp\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.221348 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.221344    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-var-lib-openvswitch\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.221500 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.221398    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6671c1c1-b674-4a1d-9011-1ca22724b234-env-overrides\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.221500 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.221455    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3a19458a-03be-4a10-a254-cec3229f5266-serviceca\") pod \"node-ca-qxmtt\" (UID: \"3a19458a-03be-4a10-a254-cec3229f5266\") " pod="openshift-image-registry/node-ca-qxmtt"
May 28 00:55:43.221500 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.221483    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/92ed3e71-738c-462d-b98e-44ae3f9a005b-device-dir\") pod \"aws-ebs-csi-driver-node-76lgg\" (UID: \"92ed3e71-738c-462d-b98e-44ae3f9a005b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg"
May 28 00:55:43.221646 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.221510    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9prq\" (UniqueName: \"kubernetes.io/projected/92ed3e71-738c-462d-b98e-44ae3f9a005b-kube-api-access-f9prq\") pod \"aws-ebs-csi-driver-node-76lgg\" (UID: \"92ed3e71-738c-462d-b98e-44ae3f9a005b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg"
May 28 00:55:43.221646 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.221571    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7zzv\" (UniqueName: \"kubernetes.io/projected/3f79c5d7-ac7c-4ef9-ada8-6d640a7c0ba7-kube-api-access-p7zzv\") pod \"node-resolver-m47th\" (UID: \"3f79c5d7-ac7c-4ef9-ada8-6d640a7c0ba7\") " pod="openshift-dns/node-resolver-m47th"
May 28 00:55:43.221646 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.221598    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-etc-systemd\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.221646 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.221625    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-etc-sysctl-d\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.221791 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.221669    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-systemd-units\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.221791 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.221699    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-host-slash\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.221791 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.221730    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-log-socket\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.221791 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.221768    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.221954 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.221796    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/92ed3e71-738c-462d-b98e-44ae3f9a005b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-76lgg\" (UID: \"92ed3e71-738c-462d-b98e-44ae3f9a005b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg"
May 28 00:55:43.221954 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.221822    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/92ed3e71-738c-462d-b98e-44ae3f9a005b-registration-dir\") pod \"aws-ebs-csi-driver-node-76lgg\" (UID: \"92ed3e71-738c-462d-b98e-44ae3f9a005b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg"
May 28 00:55:43.221954 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.221850    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3f79c5d7-ac7c-4ef9-ada8-6d640a7c0ba7-hosts-file\") pod \"node-resolver-m47th\" (UID: \"3f79c5d7-ac7c-4ef9-ada8-6d640a7c0ba7\") " pod="openshift-dns/node-resolver-m47th"
May 28 00:55:43.221954 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.221873    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-sys\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.221954 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.221911    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-lib-modules\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.221954 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.221935    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bc5kx\" (UniqueName: \"kubernetes.io/projected/ba57d944-f539-4167-be8f-dd6a88026256-kube-api-access-bc5kx\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.222223 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.221960    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d49kk\" (UniqueName: \"kubernetes.io/projected/3a19458a-03be-4a10-a254-cec3229f5266-kube-api-access-d49kk\") pod \"node-ca-qxmtt\" (UID: \"3a19458a-03be-4a10-a254-cec3229f5266\") " pod="openshift-image-registry/node-ca-qxmtt"
May 28 00:55:43.222223 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.221986    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/92ed3e71-738c-462d-b98e-44ae3f9a005b-sys-fs\") pod \"aws-ebs-csi-driver-node-76lgg\" (UID: \"92ed3e71-738c-462d-b98e-44ae3f9a005b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg"
May 28 00:55:43.222223 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222007    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-etc-openvswitch\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.222223 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222041    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-etc-kubernetes\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.222223 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222089    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-node-log\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.222223 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222115    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-host-cni-netd\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.222223 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222151    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a4ecd41b-62cb-4336-9b2b-d082dd27c847-host-slash\") pod \"iptables-alerter-nj698\" (UID: \"a4ecd41b-62cb-4336-9b2b-d082dd27c847\") " pod="openshift-network-operator/iptables-alerter-nj698"
May 28 00:55:43.222223 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222176    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f79c5d7-ac7c-4ef9-ada8-6d640a7c0ba7-tmp-dir\") pod \"node-resolver-m47th\" (UID: \"3f79c5d7-ac7c-4ef9-ada8-6d640a7c0ba7\") " pod="openshift-dns/node-resolver-m47th"
May 28 00:55:43.222223 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222204    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-host-cni-bin\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.222596 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222228    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-etc-modprobe-d\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.222596 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222245    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/92ed3e71-738c-462d-b98e-44ae3f9a005b-socket-dir\") pod \"aws-ebs-csi-driver-node-76lgg\" (UID: \"92ed3e71-738c-462d-b98e-44ae3f9a005b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg"
May 28 00:55:43.222596 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222267    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/92ed3e71-738c-462d-b98e-44ae3f9a005b-etc-selinux\") pod \"aws-ebs-csi-driver-node-76lgg\" (UID: \"92ed3e71-738c-462d-b98e-44ae3f9a005b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg"
May 28 00:55:43.222596 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222295    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppvs6\" (UniqueName: \"kubernetes.io/projected/3d04c1e7-069a-413e-a9f6-d87a8095844d-kube-api-access-ppvs6\") pod \"network-metrics-daemon-f8h8b\" (UID: \"3d04c1e7-069a-413e-a9f6-d87a8095844d\") " pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:55:43.222596 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222345    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-host-run-netns\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.222596 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222381    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r4px\" (UniqueName: \"kubernetes.io/projected/6671c1c1-b674-4a1d-9011-1ca22724b234-kube-api-access-2r4px\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.222596 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222432    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-run\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.222596 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222461    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d04c1e7-069a-413e-a9f6-d87a8095844d-metrics-certs\") pod \"network-metrics-daemon-f8h8b\" (UID: \"3d04c1e7-069a-413e-a9f6-d87a8095844d\") " pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:55:43.222596 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222481    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-run-openvswitch\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.222596 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222496    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6671c1c1-b674-4a1d-9011-1ca22724b234-ovnkube-config\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.222596 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222517    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-host-kubelet\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.222596 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222544    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-host-run-ovn-kubernetes\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.222596 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222574    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6671c1c1-b674-4a1d-9011-1ca22724b234-ovn-node-metrics-cert\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.223077 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222625    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-etc-sysconfig\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.223077 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222650    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1f856cae-503d-46e6-83f2-ef3127046b5c-agent-certs\") pod \"konnectivity-agent-lhxvp\" (UID: \"1f856cae-503d-46e6-83f2-ef3127046b5c\") " pod="kube-system/konnectivity-agent-lhxvp"
May 28 00:55:43.223077 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222674    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-var-lib-kubelet\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.223077 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222697    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-host\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.223077 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222731    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a4ecd41b-62cb-4336-9b2b-d082dd27c847-iptables-alerter-script\") pod \"iptables-alerter-nj698\" (UID: \"a4ecd41b-62cb-4336-9b2b-d082dd27c847\") " pod="openshift-network-operator/iptables-alerter-nj698"
May 28 00:55:43.223077 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222758    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-run-systemd\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.223077 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222786    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-run-ovn\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.223077 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222822    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3a19458a-03be-4a10-a254-cec3229f5266-host\") pod \"node-ca-qxmtt\" (UID: \"3a19458a-03be-4a10-a254-cec3229f5266\") " pod="openshift-image-registry/node-ca-qxmtt"
May 28 00:55:43.223077 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222847    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-etc-sysctl-conf\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.223077 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222875    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba57d944-f539-4167-be8f-dd6a88026256-etc-tuned\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.223077 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.222891    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6671c1c1-b674-4a1d-9011-1ca22724b234-ovnkube-script-lib\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.258510 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.258475    2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 00:50:42 +0000 UTC" deadline="2028-01-02 08:14:22.425831265 +0000 UTC"
May 28 00:55:43.258510 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.258499    2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14023h18m39.167334417s"
May 28 00:55:43.311801 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.311734    2550 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 28 00:55:43.323098 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.323060    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-host-cni-bin\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.323266 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.323103    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-etc-modprobe-d\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.323266 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.323162    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-host-cni-bin\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.323266 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.323214    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/92ed3e71-738c-462d-b98e-44ae3f9a005b-socket-dir\") pod \"aws-ebs-csi-driver-node-76lgg\" (UID: \"92ed3e71-738c-462d-b98e-44ae3f9a005b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg"
May 28 00:55:43.323266 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.323234    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-etc-modprobe-d\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.323532 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.323268    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/92ed3e71-738c-462d-b98e-44ae3f9a005b-etc-selinux\") pod \"aws-ebs-csi-driver-node-76lgg\" (UID: \"92ed3e71-738c-462d-b98e-44ae3f9a005b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg"
May 28 00:55:43.323532 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.323296    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ppvs6\" (UniqueName: \"kubernetes.io/projected/3d04c1e7-069a-413e-a9f6-d87a8095844d-kube-api-access-ppvs6\") pod \"network-metrics-daemon-f8h8b\" (UID: \"3d04c1e7-069a-413e-a9f6-d87a8095844d\") " pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:55:43.323532 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.323336    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-host-run-netns\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.323532 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.323362    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2r4px\" (UniqueName: \"kubernetes.io/projected/6671c1c1-b674-4a1d-9011-1ca22724b234-kube-api-access-2r4px\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.323532 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.323368    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/92ed3e71-738c-462d-b98e-44ae3f9a005b-socket-dir\") pod \"aws-ebs-csi-driver-node-76lgg\" (UID: \"92ed3e71-738c-462d-b98e-44ae3f9a005b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg"
May 28 00:55:43.323532 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.323390    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/92ed3e71-738c-462d-b98e-44ae3f9a005b-etc-selinux\") pod \"aws-ebs-csi-driver-node-76lgg\" (UID: \"92ed3e71-738c-462d-b98e-44ae3f9a005b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg"
May 28 00:55:43.323532 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.323393    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzwm8\" (UniqueName: \"kubernetes.io/projected/6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7-kube-api-access-fzwm8\") pod \"network-check-target-l9q4k\" (UID: \"6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7\") " pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:55:43.323532 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.323444    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-run\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.323532 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.323475    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d04c1e7-069a-413e-a9f6-d87a8095844d-metrics-certs\") pod \"network-metrics-daemon-f8h8b\" (UID: \"3d04c1e7-069a-413e-a9f6-d87a8095844d\") " pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:55:43.323532 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.323487    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-host-run-netns\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.323979 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.323927    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-run-openvswitch\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.324030 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.324008    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6671c1c1-b674-4a1d-9011-1ca22724b234-ovnkube-config\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.324133 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.324108    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4k2x\" (UniqueName: \"kubernetes.io/projected/8ee280af-d845-48ce-9d6e-1fd1baad857d-kube-api-access-c4k2x\") pod \"multus-additional-cni-plugins-qn2qw\" (UID: \"8ee280af-d845-48ce-9d6e-1fd1baad857d\") " pod="openshift-multus/multus-additional-cni-plugins-qn2qw"
May 28 00:55:43.324205 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.324155    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d829d8f9-26d3-4bab-b318-b59320d5a67e-cni-binary-copy\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.324259 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.324205    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-host-kubelet\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.324791 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.324760    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-run\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.324887 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.324789    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6671c1c1-b674-4a1d-9011-1ca22724b234-ovnkube-config\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.324887 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.324334    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-host-kubelet\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.324887 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.324838    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-host-run-ovn-kubernetes\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.325021 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:43.324969    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 00:55:43.325021 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.324975    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-run-openvswitch\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.325106 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.325022    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-host-run-ovn-kubernetes\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.325106 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:43.325068    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d04c1e7-069a-413e-a9f6-d87a8095844d-metrics-certs podName:3d04c1e7-069a-413e-a9f6-d87a8095844d nodeName:}" failed. No retries permitted until 2026-05-28 00:55:43.825047096 +0000 UTC m=+3.106004185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d04c1e7-069a-413e-a9f6-d87a8095844d-metrics-certs") pod "network-metrics-daemon-f8h8b" (UID: "3d04c1e7-069a-413e-a9f6-d87a8095844d") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 00:55:43.325106 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.325089    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6671c1c1-b674-4a1d-9011-1ca22724b234-ovn-node-metrics-cert\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.325242 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.325119    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-host-run-netns\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.325294 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.325250    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-etc-sysconfig\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.325294 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.325283    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-host-var-lib-cni-bin\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.325394 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.325321    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1f856cae-503d-46e6-83f2-ef3127046b5c-agent-certs\") pod \"konnectivity-agent-lhxvp\" (UID: \"1f856cae-503d-46e6-83f2-ef3127046b5c\") " pod="kube-system/konnectivity-agent-lhxvp"
May 28 00:55:43.325394 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.325389    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-var-lib-kubelet\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.325507 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.325428    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-etc-sysconfig\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.325507 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.325441    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-host\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.325507 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.325480    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a4ecd41b-62cb-4336-9b2b-d082dd27c847-iptables-alerter-script\") pod \"iptables-alerter-nj698\" (UID: \"a4ecd41b-62cb-4336-9b2b-d082dd27c847\") " pod="openshift-network-operator/iptables-alerter-nj698"
May 28 00:55:43.325630 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.325561    2550 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 28 00:55:43.326035 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.326012    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a4ecd41b-62cb-4336-9b2b-d082dd27c847-iptables-alerter-script\") pod \"iptables-alerter-nj698\" (UID: \"a4ecd41b-62cb-4336-9b2b-d082dd27c847\") " pod="openshift-network-operator/iptables-alerter-nj698"
May 28 00:55:43.326144 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.326128    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-var-lib-kubelet\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.326249 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.326208    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-host\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.326299 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.326280    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-run-systemd\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.326347 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.326321    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-run-ovn\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.326394 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.326353    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3a19458a-03be-4a10-a254-cec3229f5266-host\") pod \"node-ca-qxmtt\" (UID: \"3a19458a-03be-4a10-a254-cec3229f5266\") " pod="openshift-image-registry/node-ca-qxmtt"
May 28 00:55:43.326394 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.326382    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8ee280af-d845-48ce-9d6e-1fd1baad857d-os-release\") pod \"multus-additional-cni-plugins-qn2qw\" (UID: \"8ee280af-d845-48ce-9d6e-1fd1baad857d\") " pod="openshift-multus/multus-additional-cni-plugins-qn2qw"
May 28 00:55:43.326508 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.326435    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-etc-sysctl-conf\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.326508 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.326468    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba57d944-f539-4167-be8f-dd6a88026256-etc-tuned\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.326508 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.326496    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6671c1c1-b674-4a1d-9011-1ca22724b234-ovnkube-script-lib\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.326652 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.326529    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8ee280af-d845-48ce-9d6e-1fd1baad857d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qn2qw\" (UID: \"8ee280af-d845-48ce-9d6e-1fd1baad857d\") " pod="openshift-multus/multus-additional-cni-plugins-qn2qw"
May 28 00:55:43.326652 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.326556    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8ee280af-d845-48ce-9d6e-1fd1baad857d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qn2qw\" (UID: \"8ee280af-d845-48ce-9d6e-1fd1baad857d\") " pod="openshift-multus/multus-additional-cni-plugins-qn2qw"
May 28 00:55:43.326652 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.326587    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-os-release\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.326652 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.326617    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-multus-socket-dir-parent\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.326652 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.326649    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gm4d7\" (UniqueName: \"kubernetes.io/projected/a4ecd41b-62cb-4336-9b2b-d082dd27c847-kube-api-access-gm4d7\") pod \"iptables-alerter-nj698\" (UID: \"a4ecd41b-62cb-4336-9b2b-d082dd27c847\") " pod="openshift-network-operator/iptables-alerter-nj698"
May 28 00:55:43.326878 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.326665    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-run-ovn\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.326878 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.326682    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-host-run-k8s-cni-cncf-io\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.326878 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.326702    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-etc-sysctl-conf\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.326878 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.326770    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-host-var-lib-kubelet\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.327063 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.326997    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1f856cae-503d-46e6-83f2-ef3127046b5c-konnectivity-ca\") pod \"konnectivity-agent-lhxvp\" (UID: \"1f856cae-503d-46e6-83f2-ef3127046b5c\") " pod="kube-system/konnectivity-agent-lhxvp"
May 28 00:55:43.327063 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.327037    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba57d944-f539-4167-be8f-dd6a88026256-tmp\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.327159 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.327073    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-var-lib-openvswitch\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.327210 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.327161    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6671c1c1-b674-4a1d-9011-1ca22724b234-env-overrides\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.327210 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.327168    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6671c1c1-b674-4a1d-9011-1ca22724b234-ovnkube-script-lib\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.327639 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.327616    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6671c1c1-b674-4a1d-9011-1ca22724b234-env-overrides\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.327747 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.327672    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3a19458a-03be-4a10-a254-cec3229f5266-serviceca\") pod \"node-ca-qxmtt\" (UID: \"3a19458a-03be-4a10-a254-cec3229f5266\") " pod="openshift-image-registry/node-ca-qxmtt"
May 28 00:55:43.327747 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.327704    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-system-cni-dir\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.327852 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.327751    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3a19458a-03be-4a10-a254-cec3229f5266-host\") pod \"node-ca-qxmtt\" (UID: \"3a19458a-03be-4a10-a254-cec3229f5266\") " pod="openshift-image-registry/node-ca-qxmtt"
May 28 00:55:43.327852 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.327809    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-host-var-lib-cni-multus\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.327852 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.327846    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/92ed3e71-738c-462d-b98e-44ae3f9a005b-device-dir\") pod \"aws-ebs-csi-driver-node-76lgg\" (UID: \"92ed3e71-738c-462d-b98e-44ae3f9a005b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg"
May 28 00:55:43.327996 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.327877    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1f856cae-503d-46e6-83f2-ef3127046b5c-konnectivity-ca\") pod \"konnectivity-agent-lhxvp\" (UID: \"1f856cae-503d-46e6-83f2-ef3127046b5c\") " pod="kube-system/konnectivity-agent-lhxvp"
May 28 00:55:43.327996 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.327880    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f9prq\" (UniqueName: \"kubernetes.io/projected/92ed3e71-738c-462d-b98e-44ae3f9a005b-kube-api-access-f9prq\") pod \"aws-ebs-csi-driver-node-76lgg\" (UID: \"92ed3e71-738c-462d-b98e-44ae3f9a005b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg"
May 28 00:55:43.328267 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.328226    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-var-lib-openvswitch\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.328371 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.328301    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p7zzv\" (UniqueName: \"kubernetes.io/projected/3f79c5d7-ac7c-4ef9-ada8-6d640a7c0ba7-kube-api-access-p7zzv\") pod \"node-resolver-m47th\" (UID: \"3f79c5d7-ac7c-4ef9-ada8-6d640a7c0ba7\") " pod="openshift-dns/node-resolver-m47th"
May 28 00:55:43.328371 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.328358    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-run-systemd\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.328497 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.328449    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-cnibin\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.328497 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.328492    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-etc-systemd\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.328602 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.328526    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-etc-sysctl-d\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.328602 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.328570    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-systemd-units\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.328696 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.328636    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-host-slash\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.328696 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.328669    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-log-socket\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.328842 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.328703    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.328842 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.328736    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-multus-cni-dir\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.328842 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.328765    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-hostroot\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.328842 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.328795    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/92ed3e71-738c-462d-b98e-44ae3f9a005b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-76lgg\" (UID: \"92ed3e71-738c-462d-b98e-44ae3f9a005b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg"
May 28 00:55:43.328842 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.328828    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/92ed3e71-738c-462d-b98e-44ae3f9a005b-registration-dir\") pod \"aws-ebs-csi-driver-node-76lgg\" (UID: \"92ed3e71-738c-462d-b98e-44ae3f9a005b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg"
May 28 00:55:43.329086 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.328860    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3f79c5d7-ac7c-4ef9-ada8-6d640a7c0ba7-hosts-file\") pod \"node-resolver-m47th\" (UID: \"3f79c5d7-ac7c-4ef9-ada8-6d640a7c0ba7\") " pod="openshift-dns/node-resolver-m47th"
May 28 00:55:43.329086 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.328899    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mv7c\" (UniqueName: \"kubernetes.io/projected/d829d8f9-26d3-4bab-b318-b59320d5a67e-kube-api-access-4mv7c\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.329086 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.329016    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-sys\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.329086 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.329044    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-lib-modules\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.329086 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.329079    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bc5kx\" (UniqueName: \"kubernetes.io/projected/ba57d944-f539-4167-be8f-dd6a88026256-kube-api-access-bc5kx\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.329309 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.329114    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d49kk\" (UniqueName: \"kubernetes.io/projected/3a19458a-03be-4a10-a254-cec3229f5266-kube-api-access-d49kk\") pod \"node-ca-qxmtt\" (UID: \"3a19458a-03be-4a10-a254-cec3229f5266\") " pod="openshift-image-registry/node-ca-qxmtt"
May 28 00:55:43.329309 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.329151    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8ee280af-d845-48ce-9d6e-1fd1baad857d-cni-binary-copy\") pod \"multus-additional-cni-plugins-qn2qw\" (UID: \"8ee280af-d845-48ce-9d6e-1fd1baad857d\") " pod="openshift-multus/multus-additional-cni-plugins-qn2qw"
May 28 00:55:43.329309 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.329184    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/92ed3e71-738c-462d-b98e-44ae3f9a005b-sys-fs\") pod \"aws-ebs-csi-driver-node-76lgg\" (UID: \"92ed3e71-738c-462d-b98e-44ae3f9a005b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg"
May 28 00:55:43.329309 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.329212    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-etc-openvswitch\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.329309 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.329246    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-multus-conf-dir\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.329309 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.329278    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d829d8f9-26d3-4bab-b318-b59320d5a67e-multus-daemon-config\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.329524 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.329313    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-etc-kubernetes\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.329524 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.329347    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-node-log\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.329524 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.329388    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-host-cni-netd\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.329524 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.329435    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8ee280af-d845-48ce-9d6e-1fd1baad857d-system-cni-dir\") pod \"multus-additional-cni-plugins-qn2qw\" (UID: \"8ee280af-d845-48ce-9d6e-1fd1baad857d\") " pod="openshift-multus/multus-additional-cni-plugins-qn2qw"
May 28 00:55:43.329524 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.329472    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8ee280af-d845-48ce-9d6e-1fd1baad857d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qn2qw\" (UID: \"8ee280af-d845-48ce-9d6e-1fd1baad857d\") " pod="openshift-multus/multus-additional-cni-plugins-qn2qw"
May 28 00:55:43.329524 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.329503    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-etc-kubernetes\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.329712 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.329537    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a4ecd41b-62cb-4336-9b2b-d082dd27c847-host-slash\") pod \"iptables-alerter-nj698\" (UID: \"a4ecd41b-62cb-4336-9b2b-d082dd27c847\") " pod="openshift-network-operator/iptables-alerter-nj698"
May 28 00:55:43.329712 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.329572    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f79c5d7-ac7c-4ef9-ada8-6d640a7c0ba7-tmp-dir\") pod \"node-resolver-m47th\" (UID: \"3f79c5d7-ac7c-4ef9-ada8-6d640a7c0ba7\") " pod="openshift-dns/node-resolver-m47th"
May 28 00:55:43.329712 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.329601    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8ee280af-d845-48ce-9d6e-1fd1baad857d-cnibin\") pod \"multus-additional-cni-plugins-qn2qw\" (UID: \"8ee280af-d845-48ce-9d6e-1fd1baad857d\") " pod="openshift-multus/multus-additional-cni-plugins-qn2qw"
May 28 00:55:43.329823 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.329805    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-host-run-multus-certs\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.329942 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.329926    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-systemd-units\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.330183 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.330164    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-host-slash\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.330226 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.330179    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-etc-systemd\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.330262 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.330220    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-log-socket\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.330311 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.330276    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1f856cae-503d-46e6-83f2-ef3127046b5c-agent-certs\") pod \"konnectivity-agent-lhxvp\" (UID: \"1f856cae-503d-46e6-83f2-ef3127046b5c\") " pod="kube-system/konnectivity-agent-lhxvp"
May 28 00:55:43.330346 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.330322    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-etc-sysctl-d\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.330387 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.330373    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.330649 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.330630    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3a19458a-03be-4a10-a254-cec3229f5266-serviceca\") pod \"node-ca-qxmtt\" (UID: \"3a19458a-03be-4a10-a254-cec3229f5266\") " pod="openshift-image-registry/node-ca-qxmtt"
May 28 00:55:43.330704 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.330676    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/92ed3e71-738c-462d-b98e-44ae3f9a005b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-76lgg\" (UID: \"92ed3e71-738c-462d-b98e-44ae3f9a005b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg"
May 28 00:55:43.330737 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.330709    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/92ed3e71-738c-462d-b98e-44ae3f9a005b-device-dir\") pod \"aws-ebs-csi-driver-node-76lgg\" (UID: \"92ed3e71-738c-462d-b98e-44ae3f9a005b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg"
May 28 00:55:43.330737 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.330725    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6671c1c1-b674-4a1d-9011-1ca22724b234-ovn-node-metrics-cert\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.330794 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.330739    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3f79c5d7-ac7c-4ef9-ada8-6d640a7c0ba7-hosts-file\") pod \"node-resolver-m47th\" (UID: \"3f79c5d7-ac7c-4ef9-ada8-6d640a7c0ba7\") " pod="openshift-dns/node-resolver-m47th"
May 28 00:55:43.330794 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.330741    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/92ed3e71-738c-462d-b98e-44ae3f9a005b-registration-dir\") pod \"aws-ebs-csi-driver-node-76lgg\" (UID: \"92ed3e71-738c-462d-b98e-44ae3f9a005b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg"
May 28 00:55:43.330866 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.330787    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-etc-kubernetes\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.330866 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.330833    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-sys\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.330866 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.330851    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-etc-openvswitch\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.330866 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.330863    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/92ed3e71-738c-462d-b98e-44ae3f9a005b-sys-fs\") pod \"aws-ebs-csi-driver-node-76lgg\" (UID: \"92ed3e71-738c-462d-b98e-44ae3f9a005b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg"
May 28 00:55:43.331047 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.330990    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba57d944-f539-4167-be8f-dd6a88026256-tmp\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.331101 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.331046    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba57d944-f539-4167-be8f-dd6a88026256-lib-modules\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.331150 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.331110    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-node-log\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.331206 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.331180    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a4ecd41b-62cb-4336-9b2b-d082dd27c847-host-slash\") pod \"iptables-alerter-nj698\" (UID: \"a4ecd41b-62cb-4336-9b2b-d082dd27c847\") " pod="openshift-network-operator/iptables-alerter-nj698"
May 28 00:55:43.331287 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.331266    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6671c1c1-b674-4a1d-9011-1ca22724b234-host-cni-netd\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.331667 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.331479    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f79c5d7-ac7c-4ef9-ada8-6d640a7c0ba7-tmp-dir\") pod \"node-resolver-m47th\" (UID: \"3f79c5d7-ac7c-4ef9-ada8-6d640a7c0ba7\") " pod="openshift-dns/node-resolver-m47th"
May 28 00:55:43.334404 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.334384    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba57d944-f539-4167-be8f-dd6a88026256-etc-tuned\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.335515 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.335488    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r4px\" (UniqueName: \"kubernetes.io/projected/6671c1c1-b674-4a1d-9011-1ca22724b234-kube-api-access-2r4px\") pod \"ovnkube-node-9z98m\" (UID: \"6671c1c1-b674-4a1d-9011-1ca22724b234\") " pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.336558 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.336142    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppvs6\" (UniqueName: \"kubernetes.io/projected/3d04c1e7-069a-413e-a9f6-d87a8095844d-kube-api-access-ppvs6\") pod \"network-metrics-daemon-f8h8b\" (UID: \"3d04c1e7-069a-413e-a9f6-d87a8095844d\") " pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:55:43.337701 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.337678    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm4d7\" (UniqueName: \"kubernetes.io/projected/a4ecd41b-62cb-4336-9b2b-d082dd27c847-kube-api-access-gm4d7\") pod \"iptables-alerter-nj698\" (UID: \"a4ecd41b-62cb-4336-9b2b-d082dd27c847\") " pod="openshift-network-operator/iptables-alerter-nj698"
May 28 00:55:43.337875 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.337849    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9prq\" (UniqueName: \"kubernetes.io/projected/92ed3e71-738c-462d-b98e-44ae3f9a005b-kube-api-access-f9prq\") pod \"aws-ebs-csi-driver-node-76lgg\" (UID: \"92ed3e71-738c-462d-b98e-44ae3f9a005b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg"
May 28 00:55:43.341558 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.341534    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc5kx\" (UniqueName: \"kubernetes.io/projected/ba57d944-f539-4167-be8f-dd6a88026256-kube-api-access-bc5kx\") pod \"tuned-z8xgs\" (UID: \"ba57d944-f539-4167-be8f-dd6a88026256\") " pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.341694 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.341661    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d49kk\" (UniqueName: \"kubernetes.io/projected/3a19458a-03be-4a10-a254-cec3229f5266-kube-api-access-d49kk\") pod \"node-ca-qxmtt\" (UID: \"3a19458a-03be-4a10-a254-cec3229f5266\") " pod="openshift-image-registry/node-ca-qxmtt"
May 28 00:55:43.342053 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.342029    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7zzv\" (UniqueName: \"kubernetes.io/projected/3f79c5d7-ac7c-4ef9-ada8-6d640a7c0ba7-kube-api-access-p7zzv\") pod \"node-resolver-m47th\" (UID: \"3f79c5d7-ac7c-4ef9-ada8-6d640a7c0ba7\") " pod="openshift-dns/node-resolver-m47th"
May 28 00:55:43.400608 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.400581    2550 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 00:55:43.430658 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.430614    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-multus-conf-dir\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.430658 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.430656    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d829d8f9-26d3-4bab-b318-b59320d5a67e-multus-daemon-config\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.430887 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.430686    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8ee280af-d845-48ce-9d6e-1fd1baad857d-system-cni-dir\") pod \"multus-additional-cni-plugins-qn2qw\" (UID: \"8ee280af-d845-48ce-9d6e-1fd1baad857d\") " pod="openshift-multus/multus-additional-cni-plugins-qn2qw"
May 28 00:55:43.430887 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.430721    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-multus-conf-dir\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.430887 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.430743    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8ee280af-d845-48ce-9d6e-1fd1baad857d-system-cni-dir\") pod \"multus-additional-cni-plugins-qn2qw\" (UID: \"8ee280af-d845-48ce-9d6e-1fd1baad857d\") " pod="openshift-multus/multus-additional-cni-plugins-qn2qw"
May 28 00:55:43.430887 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.430805    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8ee280af-d845-48ce-9d6e-1fd1baad857d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qn2qw\" (UID: \"8ee280af-d845-48ce-9d6e-1fd1baad857d\") " pod="openshift-multus/multus-additional-cni-plugins-qn2qw"
May 28 00:55:43.430887 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.430848    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-etc-kubernetes\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.430887 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.430881    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8ee280af-d845-48ce-9d6e-1fd1baad857d-cnibin\") pod \"multus-additional-cni-plugins-qn2qw\" (UID: \"8ee280af-d845-48ce-9d6e-1fd1baad857d\") " pod="openshift-multus/multus-additional-cni-plugins-qn2qw"
May 28 00:55:43.431169 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.430910    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-host-run-multus-certs\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.431169 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.430929    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-etc-kubernetes\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.431169 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.430945    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fzwm8\" (UniqueName: \"kubernetes.io/projected/6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7-kube-api-access-fzwm8\") pod \"network-check-target-l9q4k\" (UID: \"6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7\") " pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:55:43.431169 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.430985    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8ee280af-d845-48ce-9d6e-1fd1baad857d-cnibin\") pod \"multus-additional-cni-plugins-qn2qw\" (UID: \"8ee280af-d845-48ce-9d6e-1fd1baad857d\") " pod="openshift-multus/multus-additional-cni-plugins-qn2qw"
May 28 00:55:43.431169 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.430987    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c4k2x\" (UniqueName: \"kubernetes.io/projected/8ee280af-d845-48ce-9d6e-1fd1baad857d-kube-api-access-c4k2x\") pod \"multus-additional-cni-plugins-qn2qw\" (UID: \"8ee280af-d845-48ce-9d6e-1fd1baad857d\") " pod="openshift-multus/multus-additional-cni-plugins-qn2qw"
May 28 00:55:43.431169 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431024    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d829d8f9-26d3-4bab-b318-b59320d5a67e-cni-binary-copy\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.431169 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431027    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8ee280af-d845-48ce-9d6e-1fd1baad857d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qn2qw\" (UID: \"8ee280af-d845-48ce-9d6e-1fd1baad857d\") " pod="openshift-multus/multus-additional-cni-plugins-qn2qw"
May 28 00:55:43.431169 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431045    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-host-run-multus-certs\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.431169 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431053    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-host-run-netns\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.431169 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431082    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-host-var-lib-cni-bin\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.431169 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431113    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8ee280af-d845-48ce-9d6e-1fd1baad857d-os-release\") pod \"multus-additional-cni-plugins-qn2qw\" (UID: \"8ee280af-d845-48ce-9d6e-1fd1baad857d\") " pod="openshift-multus/multus-additional-cni-plugins-qn2qw"
May 28 00:55:43.431169 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431143    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8ee280af-d845-48ce-9d6e-1fd1baad857d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qn2qw\" (UID: \"8ee280af-d845-48ce-9d6e-1fd1baad857d\") " pod="openshift-multus/multus-additional-cni-plugins-qn2qw"
May 28 00:55:43.431169 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431169    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8ee280af-d845-48ce-9d6e-1fd1baad857d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qn2qw\" (UID: \"8ee280af-d845-48ce-9d6e-1fd1baad857d\") " pod="openshift-multus/multus-additional-cni-plugins-qn2qw"
May 28 00:55:43.431780 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431218    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d829d8f9-26d3-4bab-b318-b59320d5a67e-multus-daemon-config\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.431780 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431225    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-os-release\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.431780 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431256    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-host-var-lib-cni-bin\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.431780 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431265    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-multus-socket-dir-parent\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.431780 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431293    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-host-run-netns\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.431780 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431303    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-host-run-k8s-cni-cncf-io\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.431780 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431329    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-host-var-lib-kubelet\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.431780 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431371    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-system-cni-dir\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.431780 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431389    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8ee280af-d845-48ce-9d6e-1fd1baad857d-os-release\") pod \"multus-additional-cni-plugins-qn2qw\" (UID: \"8ee280af-d845-48ce-9d6e-1fd1baad857d\") " pod="openshift-multus/multus-additional-cni-plugins-qn2qw"
May 28 00:55:43.431780 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431397    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-host-var-lib-cni-multus\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.431780 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431423    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-os-release\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.431780 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431459    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-cnibin\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.431780 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431469    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-host-run-k8s-cni-cncf-io\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.431780 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431478    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-host-var-lib-cni-multus\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.431780 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431492    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-multus-cni-dir\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.431780 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431516    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-hostroot\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.431780 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431529    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-host-var-lib-kubelet\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.431780 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431541    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4mv7c\" (UniqueName: \"kubernetes.io/projected/d829d8f9-26d3-4bab-b318-b59320d5a67e-kube-api-access-4mv7c\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.432583 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431570    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8ee280af-d845-48ce-9d6e-1fd1baad857d-cni-binary-copy\") pod \"multus-additional-cni-plugins-qn2qw\" (UID: \"8ee280af-d845-48ce-9d6e-1fd1baad857d\") " pod="openshift-multus/multus-additional-cni-plugins-qn2qw"
May 28 00:55:43.432583 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431669    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d829d8f9-26d3-4bab-b318-b59320d5a67e-cni-binary-copy\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.432583 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431743    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-multus-cni-dir\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.432583 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431792    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-cnibin\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.432583 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431831    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-multus-socket-dir-parent\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.432583 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431863    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-hostroot\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.432583 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.431924    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d829d8f9-26d3-4bab-b318-b59320d5a67e-system-cni-dir\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.432583 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.432033    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8ee280af-d845-48ce-9d6e-1fd1baad857d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qn2qw\" (UID: \"8ee280af-d845-48ce-9d6e-1fd1baad857d\") " pod="openshift-multus/multus-additional-cni-plugins-qn2qw"
May 28 00:55:43.432583 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.432073    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8ee280af-d845-48ce-9d6e-1fd1baad857d-cni-binary-copy\") pod \"multus-additional-cni-plugins-qn2qw\" (UID: \"8ee280af-d845-48ce-9d6e-1fd1baad857d\") " pod="openshift-multus/multus-additional-cni-plugins-qn2qw"
May 28 00:55:43.432583 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.432095    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8ee280af-d845-48ce-9d6e-1fd1baad857d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qn2qw\" (UID: \"8ee280af-d845-48ce-9d6e-1fd1baad857d\") " pod="openshift-multus/multus-additional-cni-plugins-qn2qw"
May 28 00:55:43.437141 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:43.437106    2550 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 00:55:43.437141 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:43.437135    2550 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 00:55:43.437141 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:43.437148    2550 projected.go:194] Error preparing data for projected volume kube-api-access-fzwm8 for pod openshift-network-diagnostics/network-check-target-l9q4k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 00:55:43.437490 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:43.437214    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7-kube-api-access-fzwm8 podName:6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7 nodeName:}" failed. No retries permitted until 2026-05-28 00:55:43.937196883 +0000 UTC m=+3.218153987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fzwm8" (UniqueName: "kubernetes.io/projected/6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7-kube-api-access-fzwm8") pod "network-check-target-l9q4k" (UID: "6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 00:55:43.439795 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.439733    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4k2x\" (UniqueName: \"kubernetes.io/projected/8ee280af-d845-48ce-9d6e-1fd1baad857d-kube-api-access-c4k2x\") pod \"multus-additional-cni-plugins-qn2qw\" (UID: \"8ee280af-d845-48ce-9d6e-1fd1baad857d\") " pod="openshift-multus/multus-additional-cni-plugins-qn2qw"
May 28 00:55:43.440128 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.440110    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mv7c\" (UniqueName: \"kubernetes.io/projected/d829d8f9-26d3-4bab-b318-b59320d5a67e-kube-api-access-4mv7c\") pod \"multus-vxdn6\" (UID: \"d829d8f9-26d3-4bab-b318-b59320d5a67e\") " pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.509968 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.509933    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nj698"
May 28 00:55:43.518883 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.518848    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg"
May 28 00:55:43.529612 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.529581    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-m47th"
May 28 00:55:43.533331 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.533312    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qxmtt"
May 28 00:55:43.543237 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.543203    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:55:43.549863 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.549844    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lhxvp"
May 28 00:55:43.556430 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.556396    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-z8xgs"
May 28 00:55:43.564065 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.564011    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qn2qw"
May 28 00:55:43.569618 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.569596    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vxdn6"
May 28 00:55:43.834604 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:43.834519    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d04c1e7-069a-413e-a9f6-d87a8095844d-metrics-certs\") pod \"network-metrics-daemon-f8h8b\" (UID: \"3d04c1e7-069a-413e-a9f6-d87a8095844d\") " pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:55:43.834759 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:43.834659    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 00:55:43.834759 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:43.834734    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d04c1e7-069a-413e-a9f6-d87a8095844d-metrics-certs podName:3d04c1e7-069a-413e-a9f6-d87a8095844d nodeName:}" failed. No retries permitted until 2026-05-28 00:55:44.834716849 +0000 UTC m=+4.115673922 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d04c1e7-069a-413e-a9f6-d87a8095844d-metrics-certs") pod "network-metrics-daemon-f8h8b" (UID: "3d04c1e7-069a-413e-a9f6-d87a8095844d") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 00:55:43.999696 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:43.999664    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f856cae_503d_46e6_83f2_ef3127046b5c.slice/crio-8c63a21cef984f6f66568dc9f322b1158e67b8e9d84ec8e3fefbcf2e53306610 WatchSource:0}: Error finding container 8c63a21cef984f6f66568dc9f322b1158e67b8e9d84ec8e3fefbcf2e53306610: Status 404 returned error can't find the container with id 8c63a21cef984f6f66568dc9f322b1158e67b8e9d84ec8e3fefbcf2e53306610
May 28 00:55:44.000744 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:44.000717    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92ed3e71_738c_462d_b98e_44ae3f9a005b.slice/crio-ee9612efbf01b14f83af980aff89efab88c9ca67a39bf7de46f314179e27a625 WatchSource:0}: Error finding container ee9612efbf01b14f83af980aff89efab88c9ca67a39bf7de46f314179e27a625: Status 404 returned error can't find the container with id ee9612efbf01b14f83af980aff89efab88c9ca67a39bf7de46f314179e27a625
May 28 00:55:44.001462 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:44.001395    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd829d8f9_26d3_4bab_b318_b59320d5a67e.slice/crio-2dc7ec3f12ec8cba41807e0608ceab677b00165c4d5388f77482ac046bce5981 WatchSource:0}: Error finding container 2dc7ec3f12ec8cba41807e0608ceab677b00165c4d5388f77482ac046bce5981: Status 404 returned error can't find the container with id 2dc7ec3f12ec8cba41807e0608ceab677b00165c4d5388f77482ac046bce5981
May 28 00:55:44.002348 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:44.002320    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4ecd41b_62cb_4336_9b2b_d082dd27c847.slice/crio-b6eab181d495474152cb2e9b45e61c3381582bd5de9144f5c14ee4900b36b16a WatchSource:0}: Error finding container b6eab181d495474152cb2e9b45e61c3381582bd5de9144f5c14ee4900b36b16a: Status 404 returned error can't find the container with id b6eab181d495474152cb2e9b45e61c3381582bd5de9144f5c14ee4900b36b16a
May 28 00:55:44.003344 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:44.002874    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6671c1c1_b674_4a1d_9011_1ca22724b234.slice/crio-bc928167b0c7b9e17ed9879f397e492f8d12ee4aa445ea9294737896532a3ceb WatchSource:0}: Error finding container bc928167b0c7b9e17ed9879f397e492f8d12ee4aa445ea9294737896532a3ceb: Status 404 returned error can't find the container with id bc928167b0c7b9e17ed9879f397e492f8d12ee4aa445ea9294737896532a3ceb
May 28 00:55:44.005726 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:44.005704    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba57d944_f539_4167_be8f_dd6a88026256.slice/crio-639516af6b22c401667f20e04c9e35bb8231bdadeb40ddf6dea7cfbcb89ae61a WatchSource:0}: Error finding container 639516af6b22c401667f20e04c9e35bb8231bdadeb40ddf6dea7cfbcb89ae61a: Status 404 returned error can't find the container with id 639516af6b22c401667f20e04c9e35bb8231bdadeb40ddf6dea7cfbcb89ae61a
May 28 00:55:44.007239 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:44.007218    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f79c5d7_ac7c_4ef9_ada8_6d640a7c0ba7.slice/crio-36175464c7fd9be129a194c7555c80a84d1857f92ec19d8a4452aaeac1b1d438 WatchSource:0}: Error finding container 36175464c7fd9be129a194c7555c80a84d1857f92ec19d8a4452aaeac1b1d438: Status 404 returned error can't find the container with id 36175464c7fd9be129a194c7555c80a84d1857f92ec19d8a4452aaeac1b1d438
May 28 00:55:44.010917 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:44.010892    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a19458a_03be_4a10_a254_cec3229f5266.slice/crio-ffe34bb0122fe7cc3a0284659f259aa263d1780850f50350c0f84ca0378adae8 WatchSource:0}: Error finding container ffe34bb0122fe7cc3a0284659f259aa263d1780850f50350c0f84ca0378adae8: Status 404 returned error can't find the container with id ffe34bb0122fe7cc3a0284659f259aa263d1780850f50350c0f84ca0378adae8
May 28 00:55:44.011957 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:55:44.011937    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ee280af_d845_48ce_9d6e_1fd1baad857d.slice/crio-33c62e5853bce52c7cf36505b3d150aa6bb82080efc69d5196bd9d8e429253d3 WatchSource:0}: Error finding container 33c62e5853bce52c7cf36505b3d150aa6bb82080efc69d5196bd9d8e429253d3: Status 404 returned error can't find the container with id 33c62e5853bce52c7cf36505b3d150aa6bb82080efc69d5196bd9d8e429253d3
May 28 00:55:44.035702 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:44.035556    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fzwm8\" (UniqueName: \"kubernetes.io/projected/6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7-kube-api-access-fzwm8\") pod \"network-check-target-l9q4k\" (UID: \"6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7\") " pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:55:44.035790 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:44.035710    2550 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 00:55:44.035790 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:44.035729    2550 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 00:55:44.035790 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:44.035738    2550 projected.go:194] Error preparing data for projected volume kube-api-access-fzwm8 for pod openshift-network-diagnostics/network-check-target-l9q4k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 00:55:44.035790 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:44.035786    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7-kube-api-access-fzwm8 podName:6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7 nodeName:}" failed. No retries permitted until 2026-05-28 00:55:45.035771844 +0000 UTC m=+4.316728915 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fzwm8" (UniqueName: "kubernetes.io/projected/6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7-kube-api-access-fzwm8") pod "network-check-target-l9q4k" (UID: "6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 00:55:44.258709 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:44.258618    2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 00:50:42 +0000 UTC" deadline="2028-02-13 07:56:55.760195386 +0000 UTC"
May 28 00:55:44.258709 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:44.258645    2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15031h1m11.501551957s"
May 28 00:55:44.334349 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:44.334313    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qn2qw" event={"ID":"8ee280af-d845-48ce-9d6e-1fd1baad857d","Type":"ContainerStarted","Data":"33c62e5853bce52c7cf36505b3d150aa6bb82080efc69d5196bd9d8e429253d3"}
May 28 00:55:44.335346 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:44.335316    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qxmtt" event={"ID":"3a19458a-03be-4a10-a254-cec3229f5266","Type":"ContainerStarted","Data":"ffe34bb0122fe7cc3a0284659f259aa263d1780850f50350c0f84ca0378adae8"}
May 28 00:55:44.336331 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:44.336307    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-m47th" event={"ID":"3f79c5d7-ac7c-4ef9-ada8-6d640a7c0ba7","Type":"ContainerStarted","Data":"36175464c7fd9be129a194c7555c80a84d1857f92ec19d8a4452aaeac1b1d438"}
May 28 00:55:44.337293 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:44.337273    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nj698" event={"ID":"a4ecd41b-62cb-4336-9b2b-d082dd27c847","Type":"ContainerStarted","Data":"b6eab181d495474152cb2e9b45e61c3381582bd5de9144f5c14ee4900b36b16a"}
May 28 00:55:44.338388 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:44.338354    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vxdn6" event={"ID":"d829d8f9-26d3-4bab-b318-b59320d5a67e","Type":"ContainerStarted","Data":"2dc7ec3f12ec8cba41807e0608ceab677b00165c4d5388f77482ac046bce5981"}
May 28 00:55:44.339600 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:44.339561    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg" event={"ID":"92ed3e71-738c-462d-b98e-44ae3f9a005b","Type":"ContainerStarted","Data":"ee9612efbf01b14f83af980aff89efab88c9ca67a39bf7de46f314179e27a625"}
May 28 00:55:44.340710 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:44.340685    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lhxvp" event={"ID":"1f856cae-503d-46e6-83f2-ef3127046b5c","Type":"ContainerStarted","Data":"8c63a21cef984f6f66568dc9f322b1158e67b8e9d84ec8e3fefbcf2e53306610"}
May 28 00:55:44.342647 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:44.342621    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-73.ec2.internal" event={"ID":"4c51bc3462f829dd646da56091097a15","Type":"ContainerStarted","Data":"0b1f73b2512d7ae0136bddc7d621143f1b1cb49e1e77fc68f57a69402a58448b"}
May 28 00:55:44.344075 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:44.344041    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-z8xgs" event={"ID":"ba57d944-f539-4167-be8f-dd6a88026256","Type":"ContainerStarted","Data":"639516af6b22c401667f20e04c9e35bb8231bdadeb40ddf6dea7cfbcb89ae61a"}
May 28 00:55:44.345687 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:44.345658    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9z98m" event={"ID":"6671c1c1-b674-4a1d-9011-1ca22724b234","Type":"ContainerStarted","Data":"bc928167b0c7b9e17ed9879f397e492f8d12ee4aa445ea9294737896532a3ceb"}
May 28 00:55:44.347170 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:44.347148    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-73.ec2.internal" event={"ID":"00e8ca491df3bef53c3295016d026193","Type":"ContainerStarted","Data":"8dcf92e538cd940125aa67d7d56f3a81718b5df423d9258ff3d9db4f4fc0f6f1"}
May 28 00:55:44.369542 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:44.368079    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-73.ec2.internal" podStartSLOduration=2.368065101 podStartE2EDuration="2.368065101s" podCreationTimestamp="2026-05-28 00:55:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 00:55:44.367076083 +0000 UTC m=+3.648033211" watchObservedRunningTime="2026-05-28 00:55:44.368065101 +0000 UTC m=+3.649022194"
May 28 00:55:44.843258 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:44.842588    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d04c1e7-069a-413e-a9f6-d87a8095844d-metrics-certs\") pod \"network-metrics-daemon-f8h8b\" (UID: \"3d04c1e7-069a-413e-a9f6-d87a8095844d\") " pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:55:44.843258 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:44.842753    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 00:55:44.843258 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:44.842815    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d04c1e7-069a-413e-a9f6-d87a8095844d-metrics-certs podName:3d04c1e7-069a-413e-a9f6-d87a8095844d nodeName:}" failed. No retries permitted until 2026-05-28 00:55:46.842796478 +0000 UTC m=+6.123753557 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d04c1e7-069a-413e-a9f6-d87a8095844d-metrics-certs") pod "network-metrics-daemon-f8h8b" (UID: "3d04c1e7-069a-413e-a9f6-d87a8095844d") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 00:55:45.045100 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:45.044430    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fzwm8\" (UniqueName: \"kubernetes.io/projected/6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7-kube-api-access-fzwm8\") pod \"network-check-target-l9q4k\" (UID: \"6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7\") " pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:55:45.045100 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:45.044627    2550 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 00:55:45.045100 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:45.044649    2550 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 00:55:45.045100 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:45.044663    2550 projected.go:194] Error preparing data for projected volume kube-api-access-fzwm8 for pod openshift-network-diagnostics/network-check-target-l9q4k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 00:55:45.045100 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:45.044723    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7-kube-api-access-fzwm8 podName:6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7 nodeName:}" failed. No retries permitted until 2026-05-28 00:55:47.044703406 +0000 UTC m=+6.325660477 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fzwm8" (UniqueName: "kubernetes.io/projected/6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7-kube-api-access-fzwm8") pod "network-check-target-l9q4k" (UID: "6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 00:55:45.319800 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:45.319222    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:55:45.319800 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:45.319357    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8h8b" podUID="3d04c1e7-069a-413e-a9f6-d87a8095844d"
May 28 00:55:45.321343 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:45.320450    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:55:45.321343 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:45.320553    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9q4k" podUID="6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7"
May 28 00:55:45.364444 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:45.364070    2550 generic.go:358] "Generic (PLEG): container finished" podID="00e8ca491df3bef53c3295016d026193" containerID="8dcf92e538cd940125aa67d7d56f3a81718b5df423d9258ff3d9db4f4fc0f6f1" exitCode=0
May 28 00:55:45.364777 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:45.364698    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-73.ec2.internal" event={"ID":"00e8ca491df3bef53c3295016d026193","Type":"ContainerDied","Data":"8dcf92e538cd940125aa67d7d56f3a81718b5df423d9258ff3d9db4f4fc0f6f1"}
May 28 00:55:46.374021 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:46.373984    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-73.ec2.internal" event={"ID":"00e8ca491df3bef53c3295016d026193","Type":"ContainerStarted","Data":"099837c652df25f86caa802b8de4e9894463993f16dedcd1caa4145d0e3a7ae8"}
May 28 00:55:46.393089 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:46.393034    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-73.ec2.internal" podStartSLOduration=4.393013956 podStartE2EDuration="4.393013956s" podCreationTimestamp="2026-05-28 00:55:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 00:55:46.392716364 +0000 UTC m=+5.673673458" watchObservedRunningTime="2026-05-28 00:55:46.393013956 +0000 UTC m=+5.673971051"
May 28 00:55:46.859014 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:46.858908    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d04c1e7-069a-413e-a9f6-d87a8095844d-metrics-certs\") pod \"network-metrics-daemon-f8h8b\" (UID: \"3d04c1e7-069a-413e-a9f6-d87a8095844d\") " pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:55:46.859181 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:46.859060    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 00:55:46.859181 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:46.859124    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d04c1e7-069a-413e-a9f6-d87a8095844d-metrics-certs podName:3d04c1e7-069a-413e-a9f6-d87a8095844d nodeName:}" failed. No retries permitted until 2026-05-28 00:55:50.859105592 +0000 UTC m=+10.140062685 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d04c1e7-069a-413e-a9f6-d87a8095844d-metrics-certs") pod "network-metrics-daemon-f8h8b" (UID: "3d04c1e7-069a-413e-a9f6-d87a8095844d") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 00:55:47.060224 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:47.060149    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fzwm8\" (UniqueName: \"kubernetes.io/projected/6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7-kube-api-access-fzwm8\") pod \"network-check-target-l9q4k\" (UID: \"6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7\") " pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:55:47.060450 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:47.060342    2550 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 00:55:47.060450 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:47.060365    2550 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 00:55:47.060450 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:47.060379    2550 projected.go:194] Error preparing data for projected volume kube-api-access-fzwm8 for pod openshift-network-diagnostics/network-check-target-l9q4k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 00:55:47.060664 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:47.060460    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7-kube-api-access-fzwm8 podName:6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7 nodeName:}" failed. No retries permitted until 2026-05-28 00:55:51.060441724 +0000 UTC m=+10.341398801 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fzwm8" (UniqueName: "kubernetes.io/projected/6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7-kube-api-access-fzwm8") pod "network-check-target-l9q4k" (UID: "6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 00:55:47.320327 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:47.320246    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:55:47.320507 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:47.320404    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8h8b" podUID="3d04c1e7-069a-413e-a9f6-d87a8095844d"
May 28 00:55:47.320857 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:47.320712    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:55:47.320857 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:47.320816    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9q4k" podUID="6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7"
May 28 00:55:49.321002 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:49.320457    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:55:49.321002 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:49.320639    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9q4k" podUID="6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7"
May 28 00:55:49.321568 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:49.321169    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:55:49.321568 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:49.321274    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8h8b" podUID="3d04c1e7-069a-413e-a9f6-d87a8095844d"
May 28 00:55:50.890903 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:50.890862    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d04c1e7-069a-413e-a9f6-d87a8095844d-metrics-certs\") pod \"network-metrics-daemon-f8h8b\" (UID: \"3d04c1e7-069a-413e-a9f6-d87a8095844d\") " pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:55:50.891331 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:50.891035    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 00:55:50.891331 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:50.891097    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d04c1e7-069a-413e-a9f6-d87a8095844d-metrics-certs podName:3d04c1e7-069a-413e-a9f6-d87a8095844d nodeName:}" failed. No retries permitted until 2026-05-28 00:55:58.891076802 +0000 UTC m=+18.172033877 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d04c1e7-069a-413e-a9f6-d87a8095844d-metrics-certs") pod "network-metrics-daemon-f8h8b" (UID: "3d04c1e7-069a-413e-a9f6-d87a8095844d") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 00:55:51.093596 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:51.092874    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fzwm8\" (UniqueName: \"kubernetes.io/projected/6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7-kube-api-access-fzwm8\") pod \"network-check-target-l9q4k\" (UID: \"6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7\") " pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:55:51.093596 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:51.093063    2550 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 00:55:51.093596 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:51.093087    2550 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 00:55:51.093596 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:51.093101    2550 projected.go:194] Error preparing data for projected volume kube-api-access-fzwm8 for pod openshift-network-diagnostics/network-check-target-l9q4k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 00:55:51.093596 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:51.093161    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7-kube-api-access-fzwm8 podName:6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7 nodeName:}" failed. No retries permitted until 2026-05-28 00:55:59.093142501 +0000 UTC m=+18.374099590 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fzwm8" (UniqueName: "kubernetes.io/projected/6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7-kube-api-access-fzwm8") pod "network-check-target-l9q4k" (UID: "6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 00:55:51.320755 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:51.320697    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:55:51.320958 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:51.320888    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:55:51.320958 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:51.320845    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9q4k" podUID="6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7"
May 28 00:55:51.321073 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:51.321040    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8h8b" podUID="3d04c1e7-069a-413e-a9f6-d87a8095844d"
May 28 00:55:53.319989 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:53.319834    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:55:53.319989 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:53.319880    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:55:53.319989 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:53.319983    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8h8b" podUID="3d04c1e7-069a-413e-a9f6-d87a8095844d"
May 28 00:55:53.320514 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:53.320098    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9q4k" podUID="6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7"
May 28 00:55:55.319224 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:55.319191    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:55:55.319679 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:55.319241    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:55:55.319679 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:55.319348    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8h8b" podUID="3d04c1e7-069a-413e-a9f6-d87a8095844d"
May 28 00:55:55.319679 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:55.319493    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9q4k" podUID="6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7"
May 28 00:55:57.320084 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:57.320046    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:55:57.320084 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:57.320071    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:55:57.320631 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:57.320181    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9q4k" podUID="6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7"
May 28 00:55:57.320631 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:57.320292    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8h8b" podUID="3d04c1e7-069a-413e-a9f6-d87a8095844d"
May 28 00:55:57.463779 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:57.463740    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-qw4tp"]
May 28 00:55:57.467949 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:57.467927    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:55:57.468075 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:57.468020    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw4tp" podUID="42201706-6a2e-412d-8072-15c3476baba9"
May 28 00:55:57.538335 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:57.538247    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42201706-6a2e-412d-8072-15c3476baba9-original-pull-secret\") pod \"global-pull-secret-syncer-qw4tp\" (UID: \"42201706-6a2e-412d-8072-15c3476baba9\") " pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:55:57.538335 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:57.538298    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/42201706-6a2e-412d-8072-15c3476baba9-kubelet-config\") pod \"global-pull-secret-syncer-qw4tp\" (UID: \"42201706-6a2e-412d-8072-15c3476baba9\") " pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:55:57.538335 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:57.538321    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/42201706-6a2e-412d-8072-15c3476baba9-dbus\") pod \"global-pull-secret-syncer-qw4tp\" (UID: \"42201706-6a2e-412d-8072-15c3476baba9\") " pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:55:57.639374 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:57.639338    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42201706-6a2e-412d-8072-15c3476baba9-original-pull-secret\") pod \"global-pull-secret-syncer-qw4tp\" (UID: \"42201706-6a2e-412d-8072-15c3476baba9\") " pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:55:57.639374 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:57.639376    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/42201706-6a2e-412d-8072-15c3476baba9-kubelet-config\") pod \"global-pull-secret-syncer-qw4tp\" (UID: \"42201706-6a2e-412d-8072-15c3476baba9\") " pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:55:57.639613 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:57.639397    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/42201706-6a2e-412d-8072-15c3476baba9-dbus\") pod \"global-pull-secret-syncer-qw4tp\" (UID: \"42201706-6a2e-412d-8072-15c3476baba9\") " pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:55:57.639613 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:57.639496    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/42201706-6a2e-412d-8072-15c3476baba9-kubelet-config\") pod \"global-pull-secret-syncer-qw4tp\" (UID: \"42201706-6a2e-412d-8072-15c3476baba9\") " pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:55:57.639613 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:57.639518    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 00:55:57.639613 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:57.639552    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/42201706-6a2e-412d-8072-15c3476baba9-dbus\") pod \"global-pull-secret-syncer-qw4tp\" (UID: \"42201706-6a2e-412d-8072-15c3476baba9\") " pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:55:57.639613 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:57.639594    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42201706-6a2e-412d-8072-15c3476baba9-original-pull-secret podName:42201706-6a2e-412d-8072-15c3476baba9 nodeName:}" failed. No retries permitted until 2026-05-28 00:55:58.139572711 +0000 UTC m=+17.420529783 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/42201706-6a2e-412d-8072-15c3476baba9-original-pull-secret") pod "global-pull-secret-syncer-qw4tp" (UID: "42201706-6a2e-412d-8072-15c3476baba9") : object "kube-system"/"original-pull-secret" not registered
May 28 00:55:58.143103 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:58.143065    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42201706-6a2e-412d-8072-15c3476baba9-original-pull-secret\") pod \"global-pull-secret-syncer-qw4tp\" (UID: \"42201706-6a2e-412d-8072-15c3476baba9\") " pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:55:58.143291 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:58.143241    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 00:55:58.143339 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:58.143312    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42201706-6a2e-412d-8072-15c3476baba9-original-pull-secret podName:42201706-6a2e-412d-8072-15c3476baba9 nodeName:}" failed. No retries permitted until 2026-05-28 00:55:59.143296969 +0000 UTC m=+18.424254040 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/42201706-6a2e-412d-8072-15c3476baba9-original-pull-secret") pod "global-pull-secret-syncer-qw4tp" (UID: "42201706-6a2e-412d-8072-15c3476baba9") : object "kube-system"/"original-pull-secret" not registered
May 28 00:55:58.950261 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:58.950225    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d04c1e7-069a-413e-a9f6-d87a8095844d-metrics-certs\") pod \"network-metrics-daemon-f8h8b\" (UID: \"3d04c1e7-069a-413e-a9f6-d87a8095844d\") " pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:55:58.950678 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:58.950339    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 00:55:58.950678 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:58.950429    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d04c1e7-069a-413e-a9f6-d87a8095844d-metrics-certs podName:3d04c1e7-069a-413e-a9f6-d87a8095844d nodeName:}" failed. No retries permitted until 2026-05-28 00:56:14.95039398 +0000 UTC m=+34.231351054 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d04c1e7-069a-413e-a9f6-d87a8095844d-metrics-certs") pod "network-metrics-daemon-f8h8b" (UID: "3d04c1e7-069a-413e-a9f6-d87a8095844d") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 00:55:59.151072 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:59.151029    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42201706-6a2e-412d-8072-15c3476baba9-original-pull-secret\") pod \"global-pull-secret-syncer-qw4tp\" (UID: \"42201706-6a2e-412d-8072-15c3476baba9\") " pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:55:59.151257 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:59.151085    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fzwm8\" (UniqueName: \"kubernetes.io/projected/6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7-kube-api-access-fzwm8\") pod \"network-check-target-l9q4k\" (UID: \"6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7\") " pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:55:59.151257 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:59.151211    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 00:55:59.151374 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:59.151287    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42201706-6a2e-412d-8072-15c3476baba9-original-pull-secret podName:42201706-6a2e-412d-8072-15c3476baba9 nodeName:}" failed. No retries permitted until 2026-05-28 00:56:01.151271112 +0000 UTC m=+20.432228202 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/42201706-6a2e-412d-8072-15c3476baba9-original-pull-secret") pod "global-pull-secret-syncer-qw4tp" (UID: "42201706-6a2e-412d-8072-15c3476baba9") : object "kube-system"/"original-pull-secret" not registered
May 28 00:55:59.151374 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:59.151220    2550 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 00:55:59.151374 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:59.151333    2550 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 00:55:59.151374 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:59.151355    2550 projected.go:194] Error preparing data for projected volume kube-api-access-fzwm8 for pod openshift-network-diagnostics/network-check-target-l9q4k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 00:55:59.151527 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:59.151400    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7-kube-api-access-fzwm8 podName:6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7 nodeName:}" failed. No retries permitted until 2026-05-28 00:56:15.151388711 +0000 UTC m=+34.432345782 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fzwm8" (UniqueName: "kubernetes.io/projected/6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7-kube-api-access-fzwm8") pod "network-check-target-l9q4k" (UID: "6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 00:55:59.319917 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:59.319827    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:55:59.319917 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:59.319851    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:55:59.320132 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:55:59.319827    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:55:59.320132 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:59.319967    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw4tp" podUID="42201706-6a2e-412d-8072-15c3476baba9"
May 28 00:55:59.320132 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:59.320076    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9q4k" podUID="6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7"
May 28 00:55:59.320259 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:55:59.320173    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8h8b" podUID="3d04c1e7-069a-413e-a9f6-d87a8095844d"
May 28 00:56:01.166514 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:01.166465    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42201706-6a2e-412d-8072-15c3476baba9-original-pull-secret\") pod \"global-pull-secret-syncer-qw4tp\" (UID: \"42201706-6a2e-412d-8072-15c3476baba9\") " pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:56:01.166936 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:01.166620    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 00:56:01.166936 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:01.166700    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42201706-6a2e-412d-8072-15c3476baba9-original-pull-secret podName:42201706-6a2e-412d-8072-15c3476baba9 nodeName:}" failed. No retries permitted until 2026-05-28 00:56:05.166682691 +0000 UTC m=+24.447639937 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/42201706-6a2e-412d-8072-15c3476baba9-original-pull-secret") pod "global-pull-secret-syncer-qw4tp" (UID: "42201706-6a2e-412d-8072-15c3476baba9") : object "kube-system"/"original-pull-secret" not registered
May 28 00:56:01.320833 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:01.320795    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:56:01.320993 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:01.320862    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:56:01.320993 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:01.320943    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:56:01.320993 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:01.320948    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8h8b" podUID="3d04c1e7-069a-413e-a9f6-d87a8095844d"
May 28 00:56:01.321163 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:01.321023    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9q4k" podUID="6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7"
May 28 00:56:01.321163 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:01.321085    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw4tp" podUID="42201706-6a2e-412d-8072-15c3476baba9"
May 28 00:56:02.403610 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:02.403298    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vxdn6" event={"ID":"d829d8f9-26d3-4bab-b318-b59320d5a67e","Type":"ContainerStarted","Data":"d22eabda6a7fec0084ef1884766e00cdddcddd889ad6d5ddd9f910efa0131299"}
May 28 00:56:02.404796 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:02.404771    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg" event={"ID":"92ed3e71-738c-462d-b98e-44ae3f9a005b","Type":"ContainerStarted","Data":"ea8b09d929440e1c46af7ba45f102410a297b57d5282279dd09e856e2f940d39"}
May 28 00:56:02.406211 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:02.406172    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lhxvp" event={"ID":"1f856cae-503d-46e6-83f2-ef3127046b5c","Type":"ContainerStarted","Data":"18d6670938138114261f95d51f52f286c90a2617bf6f402a917c9a8022c36b83"}
May 28 00:56:02.407618 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:02.407587    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-z8xgs" event={"ID":"ba57d944-f539-4167-be8f-dd6a88026256","Type":"ContainerStarted","Data":"7ce2eabf4c45a63cd79d7208ac24319a1798ccb37b6bdf5d8e67b57026d5a592"}
May 28 00:56:02.409109 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:02.409090    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9z98m" event={"ID":"6671c1c1-b674-4a1d-9011-1ca22724b234","Type":"ContainerStarted","Data":"166eeec7915e0427be3a1e04bf33b232459c0d75aff57acc4756c95b58654a7a"}
May 28 00:56:02.412916 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:02.412875    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qn2qw" event={"ID":"8ee280af-d845-48ce-9d6e-1fd1baad857d","Type":"ContainerStarted","Data":"508a7f845761a2e2d8a5e229a73c7465c735b63aeb67692b21f7566e3b9cd83a"}
May 28 00:56:02.414116 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:02.414091    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qxmtt" event={"ID":"3a19458a-03be-4a10-a254-cec3229f5266","Type":"ContainerStarted","Data":"52dbe12870f4ec3a8e6dfc54d42ab048f0f9f39d94c328c1d12eb654fab5fc4b"}
May 28 00:56:02.415468 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:02.415448    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-m47th" event={"ID":"3f79c5d7-ac7c-4ef9-ada8-6d640a7c0ba7","Type":"ContainerStarted","Data":"39d205dcc690b6b3012bce66bfbbadd1a6288c51c4d49f5fd3515fd19376b865"}
May 28 00:56:02.425011 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:02.424973    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-vxdn6" podStartSLOduration=3.391323603 podStartE2EDuration="21.424938293s" podCreationTimestamp="2026-05-28 00:55:41 +0000 UTC" firstStartedPulling="2026-05-28 00:55:44.004216347 +0000 UTC m=+3.285173432" lastFinishedPulling="2026-05-28 00:56:02.037831048 +0000 UTC m=+21.318788122" observedRunningTime="2026-05-28 00:56:02.42466356 +0000 UTC m=+21.705620656" watchObservedRunningTime="2026-05-28 00:56:02.424938293 +0000 UTC m=+21.705895386"
May 28 00:56:02.438535 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:02.438491    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-m47th" podStartSLOduration=3.42068786 podStartE2EDuration="21.438479269s" podCreationTimestamp="2026-05-28 00:55:41 +0000 UTC" firstStartedPulling="2026-05-28 00:55:44.009655544 +0000 UTC m=+3.290612614" lastFinishedPulling="2026-05-28 00:56:02.027446935 +0000 UTC m=+21.308404023" observedRunningTime="2026-05-28 00:56:02.438001753 +0000 UTC m=+21.718958846" watchObservedRunningTime="2026-05-28 00:56:02.438479269 +0000 UTC m=+21.719436362"
May 28 00:56:02.451160 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:02.451119    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qxmtt" podStartSLOduration=3.436583498 podStartE2EDuration="21.451104337s" podCreationTimestamp="2026-05-28 00:55:41 +0000 UTC" firstStartedPulling="2026-05-28 00:55:44.012927207 +0000 UTC m=+3.293884279" lastFinishedPulling="2026-05-28 00:56:02.027448027 +0000 UTC m=+21.308405118" observedRunningTime="2026-05-28 00:56:02.450731617 +0000 UTC m=+21.731688711" watchObservedRunningTime="2026-05-28 00:56:02.451104337 +0000 UTC m=+21.732061468"
May 28 00:56:02.467685 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:02.467630    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lhxvp" podStartSLOduration=8.224535447 podStartE2EDuration="21.46761141s" podCreationTimestamp="2026-05-28 00:55:41 +0000 UTC" firstStartedPulling="2026-05-28 00:55:44.001666258 +0000 UTC m=+3.282623332" lastFinishedPulling="2026-05-28 00:55:57.244742223 +0000 UTC m=+16.525699295" observedRunningTime="2026-05-28 00:56:02.467381622 +0000 UTC m=+21.748338772" watchObservedRunningTime="2026-05-28 00:56:02.46761141 +0000 UTC m=+21.748568505"
May 28 00:56:02.487849 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:02.487806    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-z8xgs" podStartSLOduration=3.467543513 podStartE2EDuration="21.487790798s" podCreationTimestamp="2026-05-28 00:55:41 +0000 UTC" firstStartedPulling="2026-05-28 00:55:44.009572039 +0000 UTC m=+3.290529127" lastFinishedPulling="2026-05-28 00:56:02.029819335 +0000 UTC m=+21.310776412" observedRunningTime="2026-05-28 00:56:02.487173803 +0000 UTC m=+21.768130896" watchObservedRunningTime="2026-05-28 00:56:02.487790798 +0000 UTC m=+21.768747895"
May 28 00:56:03.320170 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:03.320133    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:56:03.320366 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:03.320178    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:56:03.320366 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:03.320133    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:56:03.320366 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:03.320261    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9q4k" podUID="6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7"
May 28 00:56:03.320494 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:03.320361    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw4tp" podUID="42201706-6a2e-412d-8072-15c3476baba9"
May 28 00:56:03.320529 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:03.320488    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8h8b" podUID="3d04c1e7-069a-413e-a9f6-d87a8095844d"
May 28 00:56:03.418466 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:03.418435    2550 generic.go:358] "Generic (PLEG): container finished" podID="8ee280af-d845-48ce-9d6e-1fd1baad857d" containerID="508a7f845761a2e2d8a5e229a73c7465c735b63aeb67692b21f7566e3b9cd83a" exitCode=0
May 28 00:56:03.418888 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:03.418523    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qn2qw" event={"ID":"8ee280af-d845-48ce-9d6e-1fd1baad857d","Type":"ContainerDied","Data":"508a7f845761a2e2d8a5e229a73c7465c735b63aeb67692b21f7566e3b9cd83a"}
May 28 00:56:03.419809 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:03.419789    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nj698" event={"ID":"a4ecd41b-62cb-4336-9b2b-d082dd27c847","Type":"ContainerStarted","Data":"b45d1a6ae00633244b28eba97c89f5cdbb970db7c02feaddab007c28595908e8"}
May 28 00:56:03.422255 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:03.422239    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9z98m_6671c1c1-b674-4a1d-9011-1ca22724b234/ovn-acl-logging/0.log"
May 28 00:56:03.422561 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:03.422544    2550 generic.go:358] "Generic (PLEG): container finished" podID="6671c1c1-b674-4a1d-9011-1ca22724b234" containerID="08f2520c2f5d678eedf74eb7834f0d3301d7a007d48ee469f76cf30898bd6712" exitCode=1
May 28 00:56:03.422646 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:03.422570    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9z98m" event={"ID":"6671c1c1-b674-4a1d-9011-1ca22724b234","Type":"ContainerStarted","Data":"f55e425f39eace3f1344b095125190437ec17f908d01e031aa4c737f9a7ba0b9"}
May 28 00:56:03.422646 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:03.422590    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9z98m" event={"ID":"6671c1c1-b674-4a1d-9011-1ca22724b234","Type":"ContainerStarted","Data":"a6e9e8ffb3a73c0e7a7a46afd3adefb4054233a0855a6e9175262526986ec88e"}
May 28 00:56:03.422646 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:03.422600    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9z98m" event={"ID":"6671c1c1-b674-4a1d-9011-1ca22724b234","Type":"ContainerStarted","Data":"072d8d4d9c198b095c7d9e52fd81806f6d9fa31cb81a4f267db642bab8069f7d"}
May 28 00:56:03.422646 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:03.422608    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9z98m" event={"ID":"6671c1c1-b674-4a1d-9011-1ca22724b234","Type":"ContainerStarted","Data":"66b01c067560fed00d0b2b454af3a1c7179b87bd2a7d38db4289e7486005bb70"}
May 28 00:56:03.422646 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:03.422619    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9z98m" event={"ID":"6671c1c1-b674-4a1d-9011-1ca22724b234","Type":"ContainerDied","Data":"08f2520c2f5d678eedf74eb7834f0d3301d7a007d48ee469f76cf30898bd6712"}
May 28 00:56:03.617351 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:03.617319    2550 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 28 00:56:03.666986 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:03.666963    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lhxvp"
May 28 00:56:03.667466 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:03.667449    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lhxvp"
May 28 00:56:04.297143 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:04.297042    2550 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-28T00:56:03.61734102Z","UUID":"f2f63f25-187e-481f-aad4-f3637f6f6843","Handler":null,"Name":"","Endpoint":""}
May 28 00:56:04.299210 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:04.299186    2550 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 28 00:56:04.299347 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:04.299219    2550 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 28 00:56:04.426370 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:04.426323    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg" event={"ID":"92ed3e71-738c-462d-b98e-44ae3f9a005b","Type":"ContainerStarted","Data":"ef22cdd668d6f28affef866b28d2864861524e203aa4fabff703a5d7f205d08e"}
May 28 00:56:04.426853 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:04.426812    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lhxvp"
May 28 00:56:04.427240 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:04.427219    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lhxvp"
May 28 00:56:04.442122 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:04.442073    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nj698" podStartSLOduration=5.41829797 podStartE2EDuration="23.442060398s" podCreationTimestamp="2026-05-28 00:55:41 +0000 UTC" firstStartedPulling="2026-05-28 00:55:44.004274777 +0000 UTC m=+3.285231862" lastFinishedPulling="2026-05-28 00:56:02.028037206 +0000 UTC m=+21.308994290" observedRunningTime="2026-05-28 00:56:04.441780699 +0000 UTC m=+23.722737791" watchObservedRunningTime="2026-05-28 00:56:04.442060398 +0000 UTC m=+23.723017490"
May 28 00:56:04.796075 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:04.796042    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-m47th_3f79c5d7-ac7c-4ef9-ada8-6d640a7c0ba7/dns-node-resolver/0.log"
May 28 00:56:05.200370 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:05.200148    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42201706-6a2e-412d-8072-15c3476baba9-original-pull-secret\") pod \"global-pull-secret-syncer-qw4tp\" (UID: \"42201706-6a2e-412d-8072-15c3476baba9\") " pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:56:05.200553 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:05.200299    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 00:56:05.200553 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:05.200516    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42201706-6a2e-412d-8072-15c3476baba9-original-pull-secret podName:42201706-6a2e-412d-8072-15c3476baba9 nodeName:}" failed. No retries permitted until 2026-05-28 00:56:13.200485952 +0000 UTC m=+32.481443039 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/42201706-6a2e-412d-8072-15c3476baba9-original-pull-secret") pod "global-pull-secret-syncer-qw4tp" (UID: "42201706-6a2e-412d-8072-15c3476baba9") : object "kube-system"/"original-pull-secret" not registered
May 28 00:56:05.319777 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:05.319694    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:56:05.319777 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:05.319739    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:56:05.319997 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:05.319694    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:56:05.319997 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:05.319825    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9q4k" podUID="6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7"
May 28 00:56:05.319997 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:05.319914    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw4tp" podUID="42201706-6a2e-412d-8072-15c3476baba9"
May 28 00:56:05.320143 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:05.320015    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8h8b" podUID="3d04c1e7-069a-413e-a9f6-d87a8095844d"
May 28 00:56:05.430087 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:05.430052    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg" event={"ID":"92ed3e71-738c-462d-b98e-44ae3f9a005b","Type":"ContainerStarted","Data":"8509c82e5ce239dd3e2585abe1f0558f1bc84889424b3d6ea2802a3803cbdfba"}
May 28 00:56:05.433209 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:05.433183    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9z98m_6671c1c1-b674-4a1d-9011-1ca22724b234/ovn-acl-logging/0.log"
May 28 00:56:05.433539 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:05.433516    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9z98m" event={"ID":"6671c1c1-b674-4a1d-9011-1ca22724b234","Type":"ContainerStarted","Data":"a799b327238c5055d91c14a9824166b9691881a9fb37a9384201feca4ac2df38"}
May 28 00:56:05.448508 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:05.448461    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76lgg" podStartSLOduration=3.495286904 podStartE2EDuration="24.448447625s" podCreationTimestamp="2026-05-28 00:55:41 +0000 UTC" firstStartedPulling="2026-05-28 00:55:44.002657 +0000 UTC m=+3.283614078" lastFinishedPulling="2026-05-28 00:56:04.955817705 +0000 UTC m=+24.236774799" observedRunningTime="2026-05-28 00:56:05.448135664 +0000 UTC m=+24.729092759" watchObservedRunningTime="2026-05-28 00:56:05.448447625 +0000 UTC m=+24.729404715"
May 28 00:56:05.979007 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:05.978977    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qxmtt_3a19458a-03be-4a10-a254-cec3229f5266/node-ca/0.log"
May 28 00:56:07.319216 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:07.319181    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:56:07.319879 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:07.319181    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:56:07.319879 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:07.319309    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8h8b" podUID="3d04c1e7-069a-413e-a9f6-d87a8095844d"
May 28 00:56:07.319879 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:07.319186    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:56:07.319879 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:07.319365    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw4tp" podUID="42201706-6a2e-412d-8072-15c3476baba9"
May 28 00:56:07.319879 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:07.319491    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9q4k" podUID="6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7"
May 28 00:56:08.442106 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:08.441953    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9z98m_6671c1c1-b674-4a1d-9011-1ca22724b234/ovn-acl-logging/0.log"
May 28 00:56:08.442689 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:08.442406    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9z98m" event={"ID":"6671c1c1-b674-4a1d-9011-1ca22724b234","Type":"ContainerStarted","Data":"5ada9f40e0b7b87e39ad19dd1e1116be2dcf439e45563218b360139d352f9cf1"}
May 28 00:56:08.442774 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:08.442720    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:56:08.442774 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:08.442750    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:56:08.442868 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:08.442853    2550 scope.go:117] "RemoveContainer" containerID="08f2520c2f5d678eedf74eb7834f0d3301d7a007d48ee469f76cf30898bd6712"
May 28 00:56:08.444172 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:08.444147    2550 generic.go:358] "Generic (PLEG): container finished" podID="8ee280af-d845-48ce-9d6e-1fd1baad857d" containerID="a7bf059ca2bb939e6de56f9aaac67c2892780e1f1fd965d0917fc8fd8f29765d" exitCode=0
May 28 00:56:08.444262 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:08.444183    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qn2qw" event={"ID":"8ee280af-d845-48ce-9d6e-1fd1baad857d","Type":"ContainerDied","Data":"a7bf059ca2bb939e6de56f9aaac67c2892780e1f1fd965d0917fc8fd8f29765d"}
May 28 00:56:08.459856 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:08.459831    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:56:09.319198 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:09.319162    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:56:09.319358 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:09.319217    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:56:09.319358 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:09.319293    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9q4k" podUID="6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7"
May 28 00:56:09.319488 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:09.319380    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8h8b" podUID="3d04c1e7-069a-413e-a9f6-d87a8095844d"
May 28 00:56:09.319488 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:09.319429    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:56:09.319548 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:09.319526    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw4tp" podUID="42201706-6a2e-412d-8072-15c3476baba9"
May 28 00:56:09.448064 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:09.448029    2550 generic.go:358] "Generic (PLEG): container finished" podID="8ee280af-d845-48ce-9d6e-1fd1baad857d" containerID="6fa8e816f8a6012eaa7abce90fbeb2bc7c3d235ec56be3f37bb7412b0c3edf85" exitCode=0
May 28 00:56:09.448506 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:09.448110    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qn2qw" event={"ID":"8ee280af-d845-48ce-9d6e-1fd1baad857d","Type":"ContainerDied","Data":"6fa8e816f8a6012eaa7abce90fbeb2bc7c3d235ec56be3f37bb7412b0c3edf85"}
May 28 00:56:09.451463 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:09.451442    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9z98m_6671c1c1-b674-4a1d-9011-1ca22724b234/ovn-acl-logging/0.log"
May 28 00:56:09.451844 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:09.451818    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9z98m" event={"ID":"6671c1c1-b674-4a1d-9011-1ca22724b234","Type":"ContainerStarted","Data":"4ebd4523b70af68011fd67924678a0ac6062a00bfe1a45fcfd3be15f0f4de5c3"}
May 28 00:56:09.452104 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:09.452081    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:56:09.466928 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:09.466901    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:56:09.497227 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:09.497185    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9z98m" podStartSLOduration=10.430732107 podStartE2EDuration="28.497172454s" podCreationTimestamp="2026-05-28 00:55:41 +0000 UTC" firstStartedPulling="2026-05-28 00:55:44.00526367 +0000 UTC m=+3.286220740" lastFinishedPulling="2026-05-28 00:56:02.07170401 +0000 UTC m=+21.352661087" observedRunningTime="2026-05-28 00:56:09.497074132 +0000 UTC m=+28.778031247" watchObservedRunningTime="2026-05-28 00:56:09.497172454 +0000 UTC m=+28.778129547"
May 28 00:56:10.456392 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:10.456360    2550 generic.go:358] "Generic (PLEG): container finished" podID="8ee280af-d845-48ce-9d6e-1fd1baad857d" containerID="8dbdc144f68924c23b1edae7e067a40e641801c4d21561b226f83e78b158f93c" exitCode=0
May 28 00:56:10.456822 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:10.456447    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qn2qw" event={"ID":"8ee280af-d845-48ce-9d6e-1fd1baad857d","Type":"ContainerDied","Data":"8dbdc144f68924c23b1edae7e067a40e641801c4d21561b226f83e78b158f93c"}
May 28 00:56:11.319984 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:11.319954    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:56:11.320172 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:11.320036    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9q4k" podUID="6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7"
May 28 00:56:11.320172 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:11.320119    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:56:11.320297 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:11.320205    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:56:11.320297 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:11.320229    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw4tp" podUID="42201706-6a2e-412d-8072-15c3476baba9"
May 28 00:56:11.320297 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:11.320270    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8h8b" podUID="3d04c1e7-069a-413e-a9f6-d87a8095844d"
May 28 00:56:13.263715 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:13.263675    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42201706-6a2e-412d-8072-15c3476baba9-original-pull-secret\") pod \"global-pull-secret-syncer-qw4tp\" (UID: \"42201706-6a2e-412d-8072-15c3476baba9\") " pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:56:13.264188 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:13.263851    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 00:56:13.264188 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:13.263930    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42201706-6a2e-412d-8072-15c3476baba9-original-pull-secret podName:42201706-6a2e-412d-8072-15c3476baba9 nodeName:}" failed. No retries permitted until 2026-05-28 00:56:29.263910186 +0000 UTC m=+48.544867276 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/42201706-6a2e-412d-8072-15c3476baba9-original-pull-secret") pod "global-pull-secret-syncer-qw4tp" (UID: "42201706-6a2e-412d-8072-15c3476baba9") : object "kube-system"/"original-pull-secret" not registered
May 28 00:56:13.322629 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:13.322597    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:56:13.322792 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:13.322646    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:56:13.322792 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:13.322652    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:56:13.322792 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:13.322726    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9q4k" podUID="6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7"
May 28 00:56:13.322945 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:13.322798    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8h8b" podUID="3d04c1e7-069a-413e-a9f6-d87a8095844d"
May 28 00:56:13.322945 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:13.322861    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw4tp" podUID="42201706-6a2e-412d-8072-15c3476baba9"
May 28 00:56:14.977591 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:14.977499    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d04c1e7-069a-413e-a9f6-d87a8095844d-metrics-certs\") pod \"network-metrics-daemon-f8h8b\" (UID: \"3d04c1e7-069a-413e-a9f6-d87a8095844d\") " pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:56:14.978087 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:14.977625    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 00:56:14.978087 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:14.977708    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d04c1e7-069a-413e-a9f6-d87a8095844d-metrics-certs podName:3d04c1e7-069a-413e-a9f6-d87a8095844d nodeName:}" failed. No retries permitted until 2026-05-28 00:56:46.977685999 +0000 UTC m=+66.258643091 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d04c1e7-069a-413e-a9f6-d87a8095844d-metrics-certs") pod "network-metrics-daemon-f8h8b" (UID: "3d04c1e7-069a-413e-a9f6-d87a8095844d") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 00:56:15.179191 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:15.179156    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fzwm8\" (UniqueName: \"kubernetes.io/projected/6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7-kube-api-access-fzwm8\") pod \"network-check-target-l9q4k\" (UID: \"6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7\") " pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:56:15.179364 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:15.179333    2550 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 00:56:15.179364 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:15.179355    2550 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 00:56:15.179364 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:15.179365    2550 projected.go:194] Error preparing data for projected volume kube-api-access-fzwm8 for pod openshift-network-diagnostics/network-check-target-l9q4k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 00:56:15.179517 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:15.179435    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7-kube-api-access-fzwm8 podName:6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7 nodeName:}" failed. No retries permitted until 2026-05-28 00:56:47.179404499 +0000 UTC m=+66.460361569 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fzwm8" (UniqueName: "kubernetes.io/projected/6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7-kube-api-access-fzwm8") pod "network-check-target-l9q4k" (UID: "6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 00:56:15.319679 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:15.319588    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:56:15.319679 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:15.319642    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:56:15.319905 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:15.319718    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9q4k" podUID="6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7"
May 28 00:56:15.319905 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:15.319813    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw4tp" podUID="42201706-6a2e-412d-8072-15c3476baba9"
May 28 00:56:15.319905 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:15.319855    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:56:15.320037 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:15.320017    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8h8b" podUID="3d04c1e7-069a-413e-a9f6-d87a8095844d"
May 28 00:56:17.321552 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:17.321527    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:56:17.321912 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:17.321527    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:56:17.321912 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:17.321626    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8h8b" podUID="3d04c1e7-069a-413e-a9f6-d87a8095844d"
May 28 00:56:17.321912 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:17.321700    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9q4k" podUID="6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7"
May 28 00:56:17.321912 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:17.321532    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:56:17.321912 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:17.321770    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw4tp" podUID="42201706-6a2e-412d-8072-15c3476baba9"
May 28 00:56:17.471244 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:17.471208    2550 generic.go:358] "Generic (PLEG): container finished" podID="8ee280af-d845-48ce-9d6e-1fd1baad857d" containerID="aeeeabedfb918fac9c88ffff849375c28e17377b60ec2421c7f43e26fb99ea44" exitCode=0
May 28 00:56:17.471397 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:17.471251    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qn2qw" event={"ID":"8ee280af-d845-48ce-9d6e-1fd1baad857d","Type":"ContainerDied","Data":"aeeeabedfb918fac9c88ffff849375c28e17377b60ec2421c7f43e26fb99ea44"}
May 28 00:56:18.476696 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:18.476658    2550 generic.go:358] "Generic (PLEG): container finished" podID="8ee280af-d845-48ce-9d6e-1fd1baad857d" containerID="edc64a73035182c455112498be6ffab416f7a5d4f7942c948fc9da835a733e27" exitCode=0
May 28 00:56:18.477072 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:18.476720    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qn2qw" event={"ID":"8ee280af-d845-48ce-9d6e-1fd1baad857d","Type":"ContainerDied","Data":"edc64a73035182c455112498be6ffab416f7a5d4f7942c948fc9da835a733e27"}
May 28 00:56:19.319356 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:19.319193    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:56:19.319563 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:19.319193    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:56:19.319563 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:19.319479    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9q4k" podUID="6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7"
May 28 00:56:19.319563 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:19.319193    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:56:19.319729 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:19.319507    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw4tp" podUID="42201706-6a2e-412d-8072-15c3476baba9"
May 28 00:56:19.319729 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:19.319581    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8h8b" podUID="3d04c1e7-069a-413e-a9f6-d87a8095844d"
May 28 00:56:19.483245 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:19.483209    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qn2qw" event={"ID":"8ee280af-d845-48ce-9d6e-1fd1baad857d","Type":"ContainerStarted","Data":"b79da1ac06cdb8fbe671cf74721173f1e4b5a7de435d2481296ae2fd167b8c35"}
May 28 00:56:21.320716 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:21.320675    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:56:21.321103 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:21.320790    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9q4k" podUID="6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7"
May 28 00:56:21.321103 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:21.320804    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:56:21.321103 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:21.320826    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:56:21.321103 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:21.320955    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw4tp" podUID="42201706-6a2e-412d-8072-15c3476baba9"
May 28 00:56:21.321103 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:21.321058    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8h8b" podUID="3d04c1e7-069a-413e-a9f6-d87a8095844d"
May 28 00:56:23.319667 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:23.319632    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:56:23.319667 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:23.319651    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:56:23.320213 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:23.319633    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:56:23.320213 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:23.319737    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8h8b" podUID="3d04c1e7-069a-413e-a9f6-d87a8095844d"
May 28 00:56:23.320213 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:23.319807    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw4tp" podUID="42201706-6a2e-412d-8072-15c3476baba9"
May 28 00:56:23.320213 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:23.319890    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9q4k" podUID="6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7"
May 28 00:56:25.319893 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:25.319855    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:56:25.319893 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:25.319877    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:56:25.320391 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:25.319879    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:56:25.320391 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:25.319959    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9q4k" podUID="6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7"
May 28 00:56:25.320391 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:25.320069    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8h8b" podUID="3d04c1e7-069a-413e-a9f6-d87a8095844d"
May 28 00:56:25.320391 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:25.320170    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw4tp" podUID="42201706-6a2e-412d-8072-15c3476baba9"
May 28 00:56:26.456154 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:26.456093    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qn2qw" podStartSLOduration=13.039673937 podStartE2EDuration="45.456074602s" podCreationTimestamp="2026-05-28 00:55:41 +0000 UTC" firstStartedPulling="2026-05-28 00:55:44.013444676 +0000 UTC m=+3.294401750" lastFinishedPulling="2026-05-28 00:56:16.42984534 +0000 UTC m=+35.710802415" observedRunningTime="2026-05-28 00:56:19.503499933 +0000 UTC m=+38.784457026" watchObservedRunningTime="2026-05-28 00:56:26.456074602 +0000 UTC m=+45.737031699"
May 28 00:56:26.457017 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:26.456986    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-f8h8b"]
May 28 00:56:26.457219 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:26.457199    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:56:26.457455 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:26.457432    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8h8b" podUID="3d04c1e7-069a-413e-a9f6-d87a8095844d"
May 28 00:56:26.459277 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:26.459252    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l9q4k"]
May 28 00:56:26.459384 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:26.459342    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:56:26.459496 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:26.459471    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9q4k" podUID="6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7"
May 28 00:56:26.470294 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:26.470261    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qw4tp"]
May 28 00:56:26.470408 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:26.470373    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:56:26.470533 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:26.470509    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw4tp" podUID="42201706-6a2e-412d-8072-15c3476baba9"
May 28 00:56:28.319631 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:28.319606    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:56:28.320049 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:28.319641    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:56:28.320049 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:28.319637    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:56:28.320049 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:28.319730    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f8h8b" podUID="3d04c1e7-069a-413e-a9f6-d87a8095844d"
May 28 00:56:28.320049 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:28.319832    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qw4tp" podUID="42201706-6a2e-412d-8072-15c3476baba9"
May 28 00:56:28.320049 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:28.319888    2550 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9q4k" podUID="6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7"
May 28 00:56:29.285029 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:29.284989    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42201706-6a2e-412d-8072-15c3476baba9-original-pull-secret\") pod \"global-pull-secret-syncer-qw4tp\" (UID: \"42201706-6a2e-412d-8072-15c3476baba9\") " pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:56:29.285211 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:29.285126    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 00:56:29.285211 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:29.285189    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42201706-6a2e-412d-8072-15c3476baba9-original-pull-secret podName:42201706-6a2e-412d-8072-15c3476baba9 nodeName:}" failed. No retries permitted until 2026-05-28 00:57:01.28517258 +0000 UTC m=+80.566129655 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/42201706-6a2e-412d-8072-15c3476baba9-original-pull-secret") pod "global-pull-secret-syncer-qw4tp" (UID: "42201706-6a2e-412d-8072-15c3476baba9") : object "kube-system"/"original-pull-secret" not registered
May 28 00:56:30.067098 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.067069    2550 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-131-73.ec2.internal" event="NodeReady"
May 28 00:56:30.067498 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.067187    2550 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 28 00:56:30.100499 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.100469    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5997b765f9-fcpb7"]
May 28 00:56:30.127910 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.127881    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5997b765f9-fcpb7"]
May 28 00:56:30.127910 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.127911    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-57vdg"]
May 28 00:56:30.128095 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.128043    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:30.130431 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.130271    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 28 00:56:30.131360 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.131336    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4jxwm\""
May 28 00:56:30.131830 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.131620    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 28 00:56:30.131830 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.131631    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 28 00:56:30.135891 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.135866    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 28 00:56:30.143809 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.143787    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gbczp"]
May 28 00:56:30.143935 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.143919    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-57vdg"
May 28 00:56:30.146116 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.146094    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lsjxl\""
May 28 00:56:30.146262 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.146097    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 28 00:56:30.146359 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.146328    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 28 00:56:30.161817 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.161788    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gbczp"]
May 28 00:56:30.161817 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.161821    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-57vdg"]
May 28 00:56:30.161975 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.161836    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tvpvm"]
May 28 00:56:30.161975 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.161929    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gbczp"
May 28 00:56:30.164791 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.164766    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 28 00:56:30.164909 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.164801    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kjscp\""
May 28 00:56:30.164909 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.164852    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 28 00:56:30.165103 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.165088    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 28 00:56:30.184080 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.184044    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tvpvm"]
May 28 00:56:30.184207 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.184194    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tvpvm"
May 28 00:56:30.186296 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.186251    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 28 00:56:30.186296 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.186293    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 28 00:56:30.186477 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.186449    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 28 00:56:30.186549 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.186505    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 28 00:56:30.186804 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.186783    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-stksl\""
May 28 00:56:30.190343 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.190322    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8c2k\" (UniqueName: \"kubernetes.io/projected/703148ea-2824-4a27-ab48-9ace83cd9f1b-kube-api-access-q8c2k\") pod \"dns-default-57vdg\" (UID: \"703148ea-2824-4a27-ab48-9ace83cd9f1b\") " pod="openshift-dns/dns-default-57vdg"
May 28 00:56:30.190535 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.190508    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqvcd\" (UniqueName: \"kubernetes.io/projected/52d52ce4-3bef-48d9-af17-cd6d77829482-kube-api-access-qqvcd\") pod \"ingress-canary-gbczp\" (UID: \"52d52ce4-3bef-48d9-af17-cd6d77829482\") " pod="openshift-ingress-canary/ingress-canary-gbczp"
May 28 00:56:30.190618 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.190541    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/703148ea-2824-4a27-ab48-9ace83cd9f1b-config-volume\") pod \"dns-default-57vdg\" (UID: \"703148ea-2824-4a27-ab48-9ace83cd9f1b\") " pod="openshift-dns/dns-default-57vdg"
May 28 00:56:30.190618 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.190571    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4ff8030f-1fcd-4ac0-9e73-cb26a0e48559-image-registry-private-configuration\") pod \"image-registry-5997b765f9-fcpb7\" (UID: \"4ff8030f-1fcd-4ac0-9e73-cb26a0e48559\") " pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:30.190618 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.190598    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4ff8030f-1fcd-4ac0-9e73-cb26a0e48559-trusted-ca\") pod \"image-registry-5997b765f9-fcpb7\" (UID: \"4ff8030f-1fcd-4ac0-9e73-cb26a0e48559\") " pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:30.190762 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.190657    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4ff8030f-1fcd-4ac0-9e73-cb26a0e48559-registry-certificates\") pod \"image-registry-5997b765f9-fcpb7\" (UID: \"4ff8030f-1fcd-4ac0-9e73-cb26a0e48559\") " pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:30.190762 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.190695    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4ff8030f-1fcd-4ac0-9e73-cb26a0e48559-ca-trust-extracted\") pod \"image-registry-5997b765f9-fcpb7\" (UID: \"4ff8030f-1fcd-4ac0-9e73-cb26a0e48559\") " pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:30.190762 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.190733    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52d52ce4-3bef-48d9-af17-cd6d77829482-cert\") pod \"ingress-canary-gbczp\" (UID: \"52d52ce4-3bef-48d9-af17-cd6d77829482\") " pod="openshift-ingress-canary/ingress-canary-gbczp"
May 28 00:56:30.190762 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.190752    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/703148ea-2824-4a27-ab48-9ace83cd9f1b-tmp-dir\") pod \"dns-default-57vdg\" (UID: \"703148ea-2824-4a27-ab48-9ace83cd9f1b\") " pod="openshift-dns/dns-default-57vdg"
May 28 00:56:30.190906 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.190772    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ff8030f-1fcd-4ac0-9e73-cb26a0e48559-registry-tls\") pod \"image-registry-5997b765f9-fcpb7\" (UID: \"4ff8030f-1fcd-4ac0-9e73-cb26a0e48559\") " pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:30.190906 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.190792    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/703148ea-2824-4a27-ab48-9ace83cd9f1b-metrics-tls\") pod \"dns-default-57vdg\" (UID: \"703148ea-2824-4a27-ab48-9ace83cd9f1b\") " pod="openshift-dns/dns-default-57vdg"
May 28 00:56:30.190906 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.190807    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4ff8030f-1fcd-4ac0-9e73-cb26a0e48559-installation-pull-secrets\") pod \"image-registry-5997b765f9-fcpb7\" (UID: \"4ff8030f-1fcd-4ac0-9e73-cb26a0e48559\") " pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:30.190906 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.190822    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zt4t\" (UniqueName: \"kubernetes.io/projected/4ff8030f-1fcd-4ac0-9e73-cb26a0e48559-kube-api-access-7zt4t\") pod \"image-registry-5997b765f9-fcpb7\" (UID: \"4ff8030f-1fcd-4ac0-9e73-cb26a0e48559\") " pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:30.190906 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.190868    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ff8030f-1fcd-4ac0-9e73-cb26a0e48559-bound-sa-token\") pod \"image-registry-5997b765f9-fcpb7\" (UID: \"4ff8030f-1fcd-4ac0-9e73-cb26a0e48559\") " pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:30.291829 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.291793    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f0a88156-df80-4562-b586-8cf16d46c000-crio-socket\") pod \"insights-runtime-extractor-tvpvm\" (UID: \"f0a88156-df80-4562-b586-8cf16d46c000\") " pod="openshift-insights/insights-runtime-extractor-tvpvm"
May 28 00:56:30.291829 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.291834    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f0a88156-df80-4562-b586-8cf16d46c000-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tvpvm\" (UID: \"f0a88156-df80-4562-b586-8cf16d46c000\") " pod="openshift-insights/insights-runtime-extractor-tvpvm"
May 28 00:56:30.292059 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.291855    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f0a88156-df80-4562-b586-8cf16d46c000-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tvpvm\" (UID: \"f0a88156-df80-4562-b586-8cf16d46c000\") " pod="openshift-insights/insights-runtime-extractor-tvpvm"
May 28 00:56:30.292059 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.291888    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q8c2k\" (UniqueName: \"kubernetes.io/projected/703148ea-2824-4a27-ab48-9ace83cd9f1b-kube-api-access-q8c2k\") pod \"dns-default-57vdg\" (UID: \"703148ea-2824-4a27-ab48-9ace83cd9f1b\") " pod="openshift-dns/dns-default-57vdg"
May 28 00:56:30.292059 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.291911    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f0a88156-df80-4562-b586-8cf16d46c000-data-volume\") pod \"insights-runtime-extractor-tvpvm\" (UID: \"f0a88156-df80-4562-b586-8cf16d46c000\") " pod="openshift-insights/insights-runtime-extractor-tvpvm"
May 28 00:56:30.292059 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.291974    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qqvcd\" (UniqueName: \"kubernetes.io/projected/52d52ce4-3bef-48d9-af17-cd6d77829482-kube-api-access-qqvcd\") pod \"ingress-canary-gbczp\" (UID: \"52d52ce4-3bef-48d9-af17-cd6d77829482\") " pod="openshift-ingress-canary/ingress-canary-gbczp"
May 28 00:56:30.292254 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.292093    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/703148ea-2824-4a27-ab48-9ace83cd9f1b-config-volume\") pod \"dns-default-57vdg\" (UID: \"703148ea-2824-4a27-ab48-9ace83cd9f1b\") " pod="openshift-dns/dns-default-57vdg"
May 28 00:56:30.292254 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.292161    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4ff8030f-1fcd-4ac0-9e73-cb26a0e48559-image-registry-private-configuration\") pod \"image-registry-5997b765f9-fcpb7\" (UID: \"4ff8030f-1fcd-4ac0-9e73-cb26a0e48559\") " pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:30.292254 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.292180    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4ff8030f-1fcd-4ac0-9e73-cb26a0e48559-trusted-ca\") pod \"image-registry-5997b765f9-fcpb7\" (UID: \"4ff8030f-1fcd-4ac0-9e73-cb26a0e48559\") " pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:30.292254 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.292202    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4ff8030f-1fcd-4ac0-9e73-cb26a0e48559-registry-certificates\") pod \"image-registry-5997b765f9-fcpb7\" (UID: \"4ff8030f-1fcd-4ac0-9e73-cb26a0e48559\") " pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:30.292254 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.292219    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg89d\" (UniqueName: \"kubernetes.io/projected/f0a88156-df80-4562-b586-8cf16d46c000-kube-api-access-dg89d\") pod \"insights-runtime-extractor-tvpvm\" (UID: \"f0a88156-df80-4562-b586-8cf16d46c000\") " pod="openshift-insights/insights-runtime-extractor-tvpvm"
May 28 00:56:30.292254 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.292249    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4ff8030f-1fcd-4ac0-9e73-cb26a0e48559-ca-trust-extracted\") pod \"image-registry-5997b765f9-fcpb7\" (UID: \"4ff8030f-1fcd-4ac0-9e73-cb26a0e48559\") " pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:30.292644 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.292313    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52d52ce4-3bef-48d9-af17-cd6d77829482-cert\") pod \"ingress-canary-gbczp\" (UID: \"52d52ce4-3bef-48d9-af17-cd6d77829482\") " pod="openshift-ingress-canary/ingress-canary-gbczp"
May 28 00:56:30.292644 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.292344    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/703148ea-2824-4a27-ab48-9ace83cd9f1b-tmp-dir\") pod \"dns-default-57vdg\" (UID: \"703148ea-2824-4a27-ab48-9ace83cd9f1b\") " pod="openshift-dns/dns-default-57vdg"
May 28 00:56:30.292644 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.292370    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ff8030f-1fcd-4ac0-9e73-cb26a0e48559-registry-tls\") pod \"image-registry-5997b765f9-fcpb7\" (UID: \"4ff8030f-1fcd-4ac0-9e73-cb26a0e48559\") " pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:30.292644 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.292399    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/703148ea-2824-4a27-ab48-9ace83cd9f1b-metrics-tls\") pod \"dns-default-57vdg\" (UID: \"703148ea-2824-4a27-ab48-9ace83cd9f1b\") " pod="openshift-dns/dns-default-57vdg"
May 28 00:56:30.292644 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.292474    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4ff8030f-1fcd-4ac0-9e73-cb26a0e48559-installation-pull-secrets\") pod \"image-registry-5997b765f9-fcpb7\" (UID: \"4ff8030f-1fcd-4ac0-9e73-cb26a0e48559\") " pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:30.292644 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.292504    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7zt4t\" (UniqueName: \"kubernetes.io/projected/4ff8030f-1fcd-4ac0-9e73-cb26a0e48559-kube-api-access-7zt4t\") pod \"image-registry-5997b765f9-fcpb7\" (UID: \"4ff8030f-1fcd-4ac0-9e73-cb26a0e48559\") " pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:30.292644 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.292548    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ff8030f-1fcd-4ac0-9e73-cb26a0e48559-bound-sa-token\") pod \"image-registry-5997b765f9-fcpb7\" (UID: \"4ff8030f-1fcd-4ac0-9e73-cb26a0e48559\") " pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:30.292979 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.292652    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/703148ea-2824-4a27-ab48-9ace83cd9f1b-tmp-dir\") pod \"dns-default-57vdg\" (UID: \"703148ea-2824-4a27-ab48-9ace83cd9f1b\") " pod="openshift-dns/dns-default-57vdg"
May 28 00:56:30.292979 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.292761    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/703148ea-2824-4a27-ab48-9ace83cd9f1b-config-volume\") pod \"dns-default-57vdg\" (UID: \"703148ea-2824-4a27-ab48-9ace83cd9f1b\") " pod="openshift-dns/dns-default-57vdg"
May 28 00:56:30.293076 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.293046    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4ff8030f-1fcd-4ac0-9e73-cb26a0e48559-ca-trust-extracted\") pod \"image-registry-5997b765f9-fcpb7\" (UID: \"4ff8030f-1fcd-4ac0-9e73-cb26a0e48559\") " pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:30.293135 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.293114    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4ff8030f-1fcd-4ac0-9e73-cb26a0e48559-registry-certificates\") pod \"image-registry-5997b765f9-fcpb7\" (UID: \"4ff8030f-1fcd-4ac0-9e73-cb26a0e48559\") " pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:30.293748 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.293723    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4ff8030f-1fcd-4ac0-9e73-cb26a0e48559-trusted-ca\") pod \"image-registry-5997b765f9-fcpb7\" (UID: \"4ff8030f-1fcd-4ac0-9e73-cb26a0e48559\") " pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:30.296333 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.296168    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/703148ea-2824-4a27-ab48-9ace83cd9f1b-metrics-tls\") pod \"dns-default-57vdg\" (UID: \"703148ea-2824-4a27-ab48-9ace83cd9f1b\") " pod="openshift-dns/dns-default-57vdg"
May 28 00:56:30.296435 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.296252    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4ff8030f-1fcd-4ac0-9e73-cb26a0e48559-installation-pull-secrets\") pod \"image-registry-5997b765f9-fcpb7\" (UID: \"4ff8030f-1fcd-4ac0-9e73-cb26a0e48559\") " pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:30.296435 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.296294    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4ff8030f-1fcd-4ac0-9e73-cb26a0e48559-image-registry-private-configuration\") pod \"image-registry-5997b765f9-fcpb7\" (UID: \"4ff8030f-1fcd-4ac0-9e73-cb26a0e48559\") " pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:30.296520 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.296452    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ff8030f-1fcd-4ac0-9e73-cb26a0e48559-registry-tls\") pod \"image-registry-5997b765f9-fcpb7\" (UID: \"4ff8030f-1fcd-4ac0-9e73-cb26a0e48559\") " pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:30.296520 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.296466    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52d52ce4-3bef-48d9-af17-cd6d77829482-cert\") pod \"ingress-canary-gbczp\" (UID: \"52d52ce4-3bef-48d9-af17-cd6d77829482\") " pod="openshift-ingress-canary/ingress-canary-gbczp"
May 28 00:56:30.299403 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.299377    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ff8030f-1fcd-4ac0-9e73-cb26a0e48559-bound-sa-token\") pod \"image-registry-5997b765f9-fcpb7\" (UID: \"4ff8030f-1fcd-4ac0-9e73-cb26a0e48559\") " pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:30.299528 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.299511    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8c2k\" (UniqueName: \"kubernetes.io/projected/703148ea-2824-4a27-ab48-9ace83cd9f1b-kube-api-access-q8c2k\") pod \"dns-default-57vdg\" (UID: \"703148ea-2824-4a27-ab48-9ace83cd9f1b\") " pod="openshift-dns/dns-default-57vdg"
May 28 00:56:30.299613 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.299597    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zt4t\" (UniqueName: \"kubernetes.io/projected/4ff8030f-1fcd-4ac0-9e73-cb26a0e48559-kube-api-access-7zt4t\") pod \"image-registry-5997b765f9-fcpb7\" (UID: \"4ff8030f-1fcd-4ac0-9e73-cb26a0e48559\") " pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:30.300015 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.299999    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqvcd\" (UniqueName: \"kubernetes.io/projected/52d52ce4-3bef-48d9-af17-cd6d77829482-kube-api-access-qqvcd\") pod \"ingress-canary-gbczp\" (UID: \"52d52ce4-3bef-48d9-af17-cd6d77829482\") " pod="openshift-ingress-canary/ingress-canary-gbczp"
May 28 00:56:30.320060 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.320006    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:56:30.320139 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.320009    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:56:30.320199 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.320009    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:56:30.322262 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.322241    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 00:56:30.322384 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.322276    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 00:56:30.322384 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.322357    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-x6qxc\""
May 28 00:56:30.322701 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.322475    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g7dls\""
May 28 00:56:30.322701 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.322485    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 28 00:56:30.322701 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.322582    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 00:56:30.393586 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.393558    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f0a88156-df80-4562-b586-8cf16d46c000-data-volume\") pod \"insights-runtime-extractor-tvpvm\" (UID: \"f0a88156-df80-4562-b586-8cf16d46c000\") " pod="openshift-insights/insights-runtime-extractor-tvpvm"
May 28 00:56:30.393742 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.393601    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dg89d\" (UniqueName: \"kubernetes.io/projected/f0a88156-df80-4562-b586-8cf16d46c000-kube-api-access-dg89d\") pod \"insights-runtime-extractor-tvpvm\" (UID: \"f0a88156-df80-4562-b586-8cf16d46c000\") " pod="openshift-insights/insights-runtime-extractor-tvpvm"
May 28 00:56:30.393742 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.393676    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f0a88156-df80-4562-b586-8cf16d46c000-crio-socket\") pod \"insights-runtime-extractor-tvpvm\" (UID: \"f0a88156-df80-4562-b586-8cf16d46c000\") " pod="openshift-insights/insights-runtime-extractor-tvpvm"
May 28 00:56:30.393742 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.393702    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f0a88156-df80-4562-b586-8cf16d46c000-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tvpvm\" (UID: \"f0a88156-df80-4562-b586-8cf16d46c000\") " pod="openshift-insights/insights-runtime-extractor-tvpvm"
May 28 00:56:30.393742 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.393728    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f0a88156-df80-4562-b586-8cf16d46c000-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tvpvm\" (UID: \"f0a88156-df80-4562-b586-8cf16d46c000\") " pod="openshift-insights/insights-runtime-extractor-tvpvm"
May 28 00:56:30.393950 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.393908    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f0a88156-df80-4562-b586-8cf16d46c000-data-volume\") pod \"insights-runtime-extractor-tvpvm\" (UID: \"f0a88156-df80-4562-b586-8cf16d46c000\") " pod="openshift-insights/insights-runtime-extractor-tvpvm"
May 28 00:56:30.393950 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.393921    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f0a88156-df80-4562-b586-8cf16d46c000-crio-socket\") pod \"insights-runtime-extractor-tvpvm\" (UID: \"f0a88156-df80-4562-b586-8cf16d46c000\") " pod="openshift-insights/insights-runtime-extractor-tvpvm"
May 28 00:56:30.394214 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.394195    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f0a88156-df80-4562-b586-8cf16d46c000-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tvpvm\" (UID: \"f0a88156-df80-4562-b586-8cf16d46c000\") " pod="openshift-insights/insights-runtime-extractor-tvpvm"
May 28 00:56:30.396129 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.396111    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f0a88156-df80-4562-b586-8cf16d46c000-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tvpvm\" (UID: \"f0a88156-df80-4562-b586-8cf16d46c000\") " pod="openshift-insights/insights-runtime-extractor-tvpvm"
May 28 00:56:30.401316 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.401299    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg89d\" (UniqueName: \"kubernetes.io/projected/f0a88156-df80-4562-b586-8cf16d46c000-kube-api-access-dg89d\") pod \"insights-runtime-extractor-tvpvm\" (UID: \"f0a88156-df80-4562-b586-8cf16d46c000\") " pod="openshift-insights/insights-runtime-extractor-tvpvm"
May 28 00:56:30.439309 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.439279    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:30.454172 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.454138    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-57vdg"
May 28 00:56:30.470890 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.470852    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gbczp"
May 28 00:56:30.492449 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.492399    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tvpvm"
May 28 00:56:30.635499 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.635124    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5997b765f9-fcpb7"]
May 28 00:56:30.636049 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.636022    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-57vdg"]
May 28 00:56:30.640702 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:56:30.640650    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod703148ea_2824_4a27_ab48_9ace83cd9f1b.slice/crio-cf8283aa6a59ef33424035164524bfb59e96326695ef77a49d6c9432e142084c WatchSource:0}: Error finding container cf8283aa6a59ef33424035164524bfb59e96326695ef77a49d6c9432e142084c: Status 404 returned error can't find the container with id cf8283aa6a59ef33424035164524bfb59e96326695ef77a49d6c9432e142084c
May 28 00:56:30.641745 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.641675    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gbczp"]
May 28 00:56:30.647350 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:56:30.647322    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ff8030f_1fcd_4ac0_9e73_cb26a0e48559.slice/crio-a77274887953573819d8e6231d6d5dc8021a56663b0d68d6027ebd57f7199ff6 WatchSource:0}: Error finding container a77274887953573819d8e6231d6d5dc8021a56663b0d68d6027ebd57f7199ff6: Status 404 returned error can't find the container with id a77274887953573819d8e6231d6d5dc8021a56663b0d68d6027ebd57f7199ff6
May 28 00:56:30.659174 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:30.659125    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tvpvm"]
May 28 00:56:30.660673 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:56:30.660631    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52d52ce4_3bef_48d9_af17_cd6d77829482.slice/crio-03ff1889b8bab008e905b8cd87d395b7418b0b918a51a99bcde1acefd8346a0d WatchSource:0}: Error finding container 03ff1889b8bab008e905b8cd87d395b7418b0b918a51a99bcde1acefd8346a0d: Status 404 returned error can't find the container with id 03ff1889b8bab008e905b8cd87d395b7418b0b918a51a99bcde1acefd8346a0d
May 28 00:56:30.662855 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:56:30.662801    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0a88156_df80_4562_b586_8cf16d46c000.slice/crio-400edee393263e272f62f69c6f84d0e8bb3cf0778af6702452ccfbb7313165b3 WatchSource:0}: Error finding container 400edee393263e272f62f69c6f84d0e8bb3cf0778af6702452ccfbb7313165b3: Status 404 returned error can't find the container with id 400edee393263e272f62f69c6f84d0e8bb3cf0778af6702452ccfbb7313165b3
May 28 00:56:31.507597 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:31.507560    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5997b765f9-fcpb7" event={"ID":"4ff8030f-1fcd-4ac0-9e73-cb26a0e48559","Type":"ContainerStarted","Data":"c2eec5c1f43f903e1598fc9b4d99004cc6e49871c571cf45dcc93ebdd2ba4a68"}
May 28 00:56:31.508105 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:31.507606    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5997b765f9-fcpb7" event={"ID":"4ff8030f-1fcd-4ac0-9e73-cb26a0e48559","Type":"ContainerStarted","Data":"a77274887953573819d8e6231d6d5dc8021a56663b0d68d6027ebd57f7199ff6"}
May 28 00:56:31.508105 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:31.507826    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:31.510693 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:31.510657    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tvpvm" event={"ID":"f0a88156-df80-4562-b586-8cf16d46c000","Type":"ContainerStarted","Data":"29007af592d458b5dff1b8c3bbc70ef6e537176675f0a1cc9245af320faeafce"}
May 28 00:56:31.510836 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:31.510698    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tvpvm" event={"ID":"f0a88156-df80-4562-b586-8cf16d46c000","Type":"ContainerStarted","Data":"400edee393263e272f62f69c6f84d0e8bb3cf0778af6702452ccfbb7313165b3"}
May 28 00:56:31.512728 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:31.512702    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-57vdg" event={"ID":"703148ea-2824-4a27-ab48-9ace83cd9f1b","Type":"ContainerStarted","Data":"cf8283aa6a59ef33424035164524bfb59e96326695ef77a49d6c9432e142084c"}
May 28 00:56:31.513975 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:31.513946    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gbczp" event={"ID":"52d52ce4-3bef-48d9-af17-cd6d77829482","Type":"ContainerStarted","Data":"03ff1889b8bab008e905b8cd87d395b7418b0b918a51a99bcde1acefd8346a0d"}
May 28 00:56:31.530987 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:31.530929    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5997b765f9-fcpb7" podStartSLOduration=6.53091074 podStartE2EDuration="6.53091074s" podCreationTimestamp="2026-05-28 00:56:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 00:56:31.530050679 +0000 UTC m=+50.811007773" watchObservedRunningTime="2026-05-28 00:56:31.53091074 +0000 UTC m=+50.811867834"
May 28 00:56:32.517371 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:32.517318    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tvpvm" event={"ID":"f0a88156-df80-4562-b586-8cf16d46c000","Type":"ContainerStarted","Data":"5cc3b3c0b8edadcb85ed1f2d58e5506430f6160731c6d9ac3d30c837c2b9ea49"}
May 28 00:56:33.529542 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:33.529400    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-57vdg" event={"ID":"703148ea-2824-4a27-ab48-9ace83cd9f1b","Type":"ContainerStarted","Data":"3f8257fcd8217b1020c8d67cde0c3708007e14a365ad04318ee667160b061e59"}
May 28 00:56:33.529542 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:33.529468    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-57vdg" event={"ID":"703148ea-2824-4a27-ab48-9ace83cd9f1b","Type":"ContainerStarted","Data":"1314d9713662fa1e08e1e6e5e495da81c946094adf6f35b60bb978d8af7f7300"}
May 28 00:56:33.530320 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:33.530120    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-57vdg"
May 28 00:56:33.531958 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:33.531862    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gbczp" event={"ID":"52d52ce4-3bef-48d9-af17-cd6d77829482","Type":"ContainerStarted","Data":"89a6138bcdc08b8d9dfe053b445c37d7142764fd3fc12d794286517e172489bb"}
May 28 00:56:33.565378 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:33.565320    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gbczp" podStartSLOduration=0.991602108 podStartE2EDuration="3.565301073s" podCreationTimestamp="2026-05-28 00:56:30 +0000 UTC" firstStartedPulling="2026-05-28 00:56:30.664631635 +0000 UTC m=+49.945588721" lastFinishedPulling="2026-05-28 00:56:33.238330616 +0000 UTC m=+52.519287686" observedRunningTime="2026-05-28 00:56:33.564987944 +0000 UTC m=+52.845945038" watchObservedRunningTime="2026-05-28 00:56:33.565301073 +0000 UTC m=+52.846258162"
May 28 00:56:33.566434 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:33.566377    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-57vdg" podStartSLOduration=0.977864734 podStartE2EDuration="3.566366263s" podCreationTimestamp="2026-05-28 00:56:30 +0000 UTC" firstStartedPulling="2026-05-28 00:56:30.644192848 +0000 UTC m=+49.925149926" lastFinishedPulling="2026-05-28 00:56:33.232694269 +0000 UTC m=+52.513651455" observedRunningTime="2026-05-28 00:56:33.54906611 +0000 UTC m=+52.830023214" watchObservedRunningTime="2026-05-28 00:56:33.566366263 +0000 UTC m=+52.847323356"
May 28 00:56:33.992836 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:33.992807    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-4w7vg"]
May 28 00:56:34.000773 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.000750    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.003046 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.003020    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-twsz6\""
May 28 00:56:34.003196 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.003054    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 28 00:56:34.003196 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.003119    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 28 00:56:34.003196 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.003133    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 28 00:56:34.003196 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.003066    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 28 00:56:34.003405 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.003216    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 28 00:56:34.003527 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.003507    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 28 00:56:34.126483 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.126448    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e1a012c0-f76b-451c-b9fc-cd5755adebdd-node-exporter-textfile\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.126483 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.126489    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e1a012c0-f76b-451c-b9fc-cd5755adebdd-node-exporter-tls\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.126690 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.126510    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1a012c0-f76b-451c-b9fc-cd5755adebdd-metrics-client-ca\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.126690 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.126529    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e1a012c0-f76b-451c-b9fc-cd5755adebdd-root\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.126690 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.126641    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1a012c0-f76b-451c-b9fc-cd5755adebdd-sys\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.126798 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.126689    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e1a012c0-f76b-451c-b9fc-cd5755adebdd-node-exporter-wtmp\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.126798 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.126717    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e1a012c0-f76b-451c-b9fc-cd5755adebdd-node-exporter-accelerators-collector-config\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.126798 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.126737    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82hm2\" (UniqueName: \"kubernetes.io/projected/e1a012c0-f76b-451c-b9fc-cd5755adebdd-kube-api-access-82hm2\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.126798 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.126756    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e1a012c0-f76b-451c-b9fc-cd5755adebdd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.227699 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.227668    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e1a012c0-f76b-451c-b9fc-cd5755adebdd-node-exporter-textfile\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.227805 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.227706    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e1a012c0-f76b-451c-b9fc-cd5755adebdd-node-exporter-tls\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.227805 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.227724    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1a012c0-f76b-451c-b9fc-cd5755adebdd-metrics-client-ca\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.227805 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.227740    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e1a012c0-f76b-451c-b9fc-cd5755adebdd-root\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.227805 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.227773    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1a012c0-f76b-451c-b9fc-cd5755adebdd-sys\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.227990 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.227811    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e1a012c0-f76b-451c-b9fc-cd5755adebdd-node-exporter-wtmp\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.227990 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.227827    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e1a012c0-f76b-451c-b9fc-cd5755adebdd-root\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.227990 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.227838    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e1a012c0-f76b-451c-b9fc-cd5755adebdd-node-exporter-accelerators-collector-config\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.227990 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.227868    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-82hm2\" (UniqueName: \"kubernetes.io/projected/e1a012c0-f76b-451c-b9fc-cd5755adebdd-kube-api-access-82hm2\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.227990 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.227873    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1a012c0-f76b-451c-b9fc-cd5755adebdd-sys\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.227990 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.227901    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e1a012c0-f76b-451c-b9fc-cd5755adebdd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.227990 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.227960    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e1a012c0-f76b-451c-b9fc-cd5755adebdd-node-exporter-wtmp\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.227990 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:34.227972    2550 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
May 28 00:56:34.228327 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.228025    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e1a012c0-f76b-451c-b9fc-cd5755adebdd-node-exporter-textfile\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.228327 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:34.228041    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1a012c0-f76b-451c-b9fc-cd5755adebdd-node-exporter-tls podName:e1a012c0-f76b-451c-b9fc-cd5755adebdd nodeName:}" failed. No retries permitted until 2026-05-28 00:56:34.7280193 +0000 UTC m=+54.008976391 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e1a012c0-f76b-451c-b9fc-cd5755adebdd-node-exporter-tls") pod "node-exporter-4w7vg" (UID: "e1a012c0-f76b-451c-b9fc-cd5755adebdd") : secret "node-exporter-tls" not found
May 28 00:56:34.228327 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.228277    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1a012c0-f76b-451c-b9fc-cd5755adebdd-metrics-client-ca\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.228450 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.228387    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e1a012c0-f76b-451c-b9fc-cd5755adebdd-node-exporter-accelerators-collector-config\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.231742 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.231725    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e1a012c0-f76b-451c-b9fc-cd5755adebdd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.237587 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.237568    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82hm2\" (UniqueName: \"kubernetes.io/projected/e1a012c0-f76b-451c-b9fc-cd5755adebdd-kube-api-access-82hm2\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.537440 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.537324    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tvpvm" event={"ID":"f0a88156-df80-4562-b586-8cf16d46c000","Type":"ContainerStarted","Data":"52f7f3aad3cd4afbbe731c6e0b753d18df4da290b1371143e875640105aab0ef"}
May 28 00:56:34.556258 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.556212    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tvpvm" podStartSLOduration=1.13868541 podStartE2EDuration="4.556198426s" podCreationTimestamp="2026-05-28 00:56:30 +0000 UTC" firstStartedPulling="2026-05-28 00:56:30.778683236 +0000 UTC m=+50.059640307" lastFinishedPulling="2026-05-28 00:56:34.196196238 +0000 UTC m=+53.477153323" observedRunningTime="2026-05-28 00:56:34.555827065 +0000 UTC m=+53.836784171" watchObservedRunningTime="2026-05-28 00:56:34.556198426 +0000 UTC m=+53.837155497"
May 28 00:56:34.732693 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.732652    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e1a012c0-f76b-451c-b9fc-cd5755adebdd-node-exporter-tls\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.735075 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.735046    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e1a012c0-f76b-451c-b9fc-cd5755adebdd-node-exporter-tls\") pod \"node-exporter-4w7vg\" (UID: \"e1a012c0-f76b-451c-b9fc-cd5755adebdd\") " pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.911929 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:34.911848    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4w7vg"
May 28 00:56:34.920498 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:56:34.920455    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1a012c0_f76b_451c_b9fc_cd5755adebdd.slice/crio-1d40ac9b20420e536742e6049affd484ac3a4e4a0b1d5a0f214346042ab638a5 WatchSource:0}: Error finding container 1d40ac9b20420e536742e6049affd484ac3a4e4a0b1d5a0f214346042ab638a5: Status 404 returned error can't find the container with id 1d40ac9b20420e536742e6049affd484ac3a4e4a0b1d5a0f214346042ab638a5
May 28 00:56:35.053090 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.053055    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 00:56:35.082425 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.082396    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 00:56:35.082584 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.082567    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.084727 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.084704    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
May 28 00:56:35.084853 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.084725    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
May 28 00:56:35.084853 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.084752    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-dknrw\""
May 28 00:56:35.085005 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.084986    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
May 28 00:56:35.085198 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.085021    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
May 28 00:56:35.085198 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.085185    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
May 28 00:56:35.085332 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.085187    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
May 28 00:56:35.085332 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.085261    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
May 28 00:56:35.085332 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.085275    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
May 28 00:56:35.085332 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.085276    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
May 28 00:56:35.136405 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.136381    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d0f24db-4a2d-4454-a9c2-5d87a855f953-config-out\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.136544 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.136436    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d0f24db-4a2d-4454-a9c2-5d87a855f953-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.136544 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.136459    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.136544 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.136481    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.136544 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.136503    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d0f24db-4a2d-4454-a9c2-5d87a855f953-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.136544 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.136533    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-576jz\" (UniqueName: \"kubernetes.io/projected/2d0f24db-4a2d-4454-a9c2-5d87a855f953-kube-api-access-576jz\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.136730 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.136573    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-web-config\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.136730 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.136594    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2d0f24db-4a2d-4454-a9c2-5d87a855f953-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.136730 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.136614    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.136730 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.136684    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.136730 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.136726    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-config-volume\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.136875 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.136747    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.136875 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.136768    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d0f24db-4a2d-4454-a9c2-5d87a855f953-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.237177 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.237141    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d0f24db-4a2d-4454-a9c2-5d87a855f953-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.237341 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.237211    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d0f24db-4a2d-4454-a9c2-5d87a855f953-config-out\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.237341 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.237254    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d0f24db-4a2d-4454-a9c2-5d87a855f953-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.237341 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.237284    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.237341 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.237320    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.237587 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.237356    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d0f24db-4a2d-4454-a9c2-5d87a855f953-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.237587 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.237385    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-576jz\" (UniqueName: \"kubernetes.io/projected/2d0f24db-4a2d-4454-a9c2-5d87a855f953-kube-api-access-576jz\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.237587 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.237434    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-web-config\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.237587 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:56:35.237533    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2d0f24db-4a2d-4454-a9c2-5d87a855f953-alertmanager-trusted-ca-bundle podName:2d0f24db-4a2d-4454-a9c2-5d87a855f953 nodeName:}" failed. No retries permitted until 2026-05-28 00:56:35.737511365 +0000 UTC m=+55.018468449 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/2d0f24db-4a2d-4454-a9c2-5d87a855f953-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "2d0f24db-4a2d-4454-a9c2-5d87a855f953") : configmap references non-existent config key: ca-bundle.crt
May 28 00:56:35.237788 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.237635    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2d0f24db-4a2d-4454-a9c2-5d87a855f953-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.237788 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.237669    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.237788 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.237718    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.239403 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.238400    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-config-volume\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.239403 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.238658    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.239973 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.239943    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d0f24db-4a2d-4454-a9c2-5d87a855f953-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.240776 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.240318    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2d0f24db-4a2d-4454-a9c2-5d87a855f953-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.241796 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.241736    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.242264 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.242221    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.242476 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.242453    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-config-volume\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.242562 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.242493    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.243125 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.243103    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-web-config\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.244571 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.243568    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d0f24db-4a2d-4454-a9c2-5d87a855f953-config-out\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.244571 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.243932    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.244970 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.244911    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d0f24db-4a2d-4454-a9c2-5d87a855f953-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.245606 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.245582    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.247483 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.247463    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-576jz\" (UniqueName: \"kubernetes.io/projected/2d0f24db-4a2d-4454-a9c2-5d87a855f953-kube-api-access-576jz\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.541284 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.541197    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4w7vg" event={"ID":"e1a012c0-f76b-451c-b9fc-cd5755adebdd","Type":"ContainerStarted","Data":"1d40ac9b20420e536742e6049affd484ac3a4e4a0b1d5a0f214346042ab638a5"}
May 28 00:56:35.743170 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.743127    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d0f24db-4a2d-4454-a9c2-5d87a855f953-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.744022 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.743990    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d0f24db-4a2d-4454-a9c2-5d87a855f953-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:35.991424 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:35.991377    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 28 00:56:36.153212 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:36.153185    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 00:56:36.157002 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:56:36.156958    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d0f24db_4a2d_4454_a9c2_5d87a855f953.slice/crio-8d30c7d4a7a440c7c2fe02347c835db2af8ef5063591b876fbe3dc2d49500d48 WatchSource:0}: Error finding container 8d30c7d4a7a440c7c2fe02347c835db2af8ef5063591b876fbe3dc2d49500d48: Status 404 returned error can't find the container with id 8d30c7d4a7a440c7c2fe02347c835db2af8ef5063591b876fbe3dc2d49500d48
May 28 00:56:36.544770 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:36.544730    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d0f24db-4a2d-4454-a9c2-5d87a855f953","Type":"ContainerStarted","Data":"8d30c7d4a7a440c7c2fe02347c835db2af8ef5063591b876fbe3dc2d49500d48"}
May 28 00:56:36.546058 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:36.546030    2550 generic.go:358] "Generic (PLEG): container finished" podID="e1a012c0-f76b-451c-b9fc-cd5755adebdd" containerID="a92c0c4c0c1a68c5658703182a76857b433fd85b9906cc41d9b23fb558b16a73" exitCode=0
May 28 00:56:36.546139 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:36.546094    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4w7vg" event={"ID":"e1a012c0-f76b-451c-b9fc-cd5755adebdd","Type":"ContainerDied","Data":"a92c0c4c0c1a68c5658703182a76857b433fd85b9906cc41d9b23fb558b16a73"}
May 28 00:56:37.552606 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:37.552571    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4w7vg" event={"ID":"e1a012c0-f76b-451c-b9fc-cd5755adebdd","Type":"ContainerStarted","Data":"c6bbab5e40e80a29924f105b33a9f47d3cd613e63bbe2642e61a0c88883abcbf"}
May 28 00:56:37.552990 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:37.552607    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4w7vg" event={"ID":"e1a012c0-f76b-451c-b9fc-cd5755adebdd","Type":"ContainerStarted","Data":"f6309a0f3f542390b92e9585027114b50fae0e86b8bcf71f218cd3b257e6fe90"}
May 28 00:56:37.575317 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:37.575273    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-4w7vg" podStartSLOduration=3.480436004 podStartE2EDuration="4.575258948s" podCreationTimestamp="2026-05-28 00:56:33 +0000 UTC" firstStartedPulling="2026-05-28 00:56:34.923248064 +0000 UTC m=+54.204205135" lastFinishedPulling="2026-05-28 00:56:36.018071005 +0000 UTC m=+55.299028079" observedRunningTime="2026-05-28 00:56:37.574810083 +0000 UTC m=+56.855767176" watchObservedRunningTime="2026-05-28 00:56:37.575258948 +0000 UTC m=+56.856216041"
May 28 00:56:38.493238 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.493201    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"]
May 28 00:56:38.513672 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.513644    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"]
May 28 00:56:38.513821 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.513765    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:56:38.515951 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.515922    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
May 28 00:56:38.516090 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.515922    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
May 28 00:56:38.516090 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.515988    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-97smk\""
May 28 00:56:38.516202 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.516193    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
May 28 00:56:38.516622 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.516595    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
May 28 00:56:38.516724 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.516622    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-73h6ufm2fflhj\""
May 28 00:56:38.556833 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.556794    2550 generic.go:358] "Generic (PLEG): container finished" podID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerID="981d77e03484b01e7da205fea04ce8ed7ec62503b70687d28875ab003e1c047c" exitCode=0
May 28 00:56:38.557245 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.556835    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d0f24db-4a2d-4454-a9c2-5d87a855f953","Type":"ContainerDied","Data":"981d77e03484b01e7da205fea04ce8ed7ec62503b70687d28875ab003e1c047c"}
May 28 00:56:38.568103 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.568077    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/afe0dd73-a436-4f93-9420-fdd562172910-client-ca-bundle\") pod \"metrics-server-75b8cbf7b5-jmflm\" (UID: \"afe0dd73-a436-4f93-9420-fdd562172910\") " pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:56:38.568226 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.568112    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/afe0dd73-a436-4f93-9420-fdd562172910-metrics-server-audit-profiles\") pod \"metrics-server-75b8cbf7b5-jmflm\" (UID: \"afe0dd73-a436-4f93-9420-fdd562172910\") " pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:56:38.568226 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.568136    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n92tk\" (UniqueName: \"kubernetes.io/projected/afe0dd73-a436-4f93-9420-fdd562172910-kube-api-access-n92tk\") pod \"metrics-server-75b8cbf7b5-jmflm\" (UID: \"afe0dd73-a436-4f93-9420-fdd562172910\") " pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:56:38.568226 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.568154    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/afe0dd73-a436-4f93-9420-fdd562172910-secret-metrics-server-client-certs\") pod \"metrics-server-75b8cbf7b5-jmflm\" (UID: \"afe0dd73-a436-4f93-9420-fdd562172910\") " pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:56:38.568226 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.568202    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/afe0dd73-a436-4f93-9420-fdd562172910-audit-log\") pod \"metrics-server-75b8cbf7b5-jmflm\" (UID: \"afe0dd73-a436-4f93-9420-fdd562172910\") " pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:56:38.568379 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.568281    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/afe0dd73-a436-4f93-9420-fdd562172910-secret-metrics-server-tls\") pod \"metrics-server-75b8cbf7b5-jmflm\" (UID: \"afe0dd73-a436-4f93-9420-fdd562172910\") " pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:56:38.568379 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.568331    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afe0dd73-a436-4f93-9420-fdd562172910-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-75b8cbf7b5-jmflm\" (UID: \"afe0dd73-a436-4f93-9420-fdd562172910\") " pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:56:38.668858 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.668817    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/afe0dd73-a436-4f93-9420-fdd562172910-client-ca-bundle\") pod \"metrics-server-75b8cbf7b5-jmflm\" (UID: \"afe0dd73-a436-4f93-9420-fdd562172910\") " pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:56:38.669052 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.668881    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/afe0dd73-a436-4f93-9420-fdd562172910-metrics-server-audit-profiles\") pod \"metrics-server-75b8cbf7b5-jmflm\" (UID: \"afe0dd73-a436-4f93-9420-fdd562172910\") " pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:56:38.669052 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.668931    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n92tk\" (UniqueName: \"kubernetes.io/projected/afe0dd73-a436-4f93-9420-fdd562172910-kube-api-access-n92tk\") pod \"metrics-server-75b8cbf7b5-jmflm\" (UID: \"afe0dd73-a436-4f93-9420-fdd562172910\") " pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:56:38.669052 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.668959    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/afe0dd73-a436-4f93-9420-fdd562172910-secret-metrics-server-client-certs\") pod \"metrics-server-75b8cbf7b5-jmflm\" (UID: \"afe0dd73-a436-4f93-9420-fdd562172910\") " pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:56:38.669052 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.668984    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/afe0dd73-a436-4f93-9420-fdd562172910-audit-log\") pod \"metrics-server-75b8cbf7b5-jmflm\" (UID: \"afe0dd73-a436-4f93-9420-fdd562172910\") " pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:56:38.669563 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.669533    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/afe0dd73-a436-4f93-9420-fdd562172910-secret-metrics-server-tls\") pod \"metrics-server-75b8cbf7b5-jmflm\" (UID: \"afe0dd73-a436-4f93-9420-fdd562172910\") " pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:56:38.669714 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.669681    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/afe0dd73-a436-4f93-9420-fdd562172910-audit-log\") pod \"metrics-server-75b8cbf7b5-jmflm\" (UID: \"afe0dd73-a436-4f93-9420-fdd562172910\") " pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:56:38.669846 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.669822    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afe0dd73-a436-4f93-9420-fdd562172910-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-75b8cbf7b5-jmflm\" (UID: \"afe0dd73-a436-4f93-9420-fdd562172910\") " pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:56:38.670178 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.670123    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/afe0dd73-a436-4f93-9420-fdd562172910-metrics-server-audit-profiles\") pod \"metrics-server-75b8cbf7b5-jmflm\" (UID: \"afe0dd73-a436-4f93-9420-fdd562172910\") " pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:56:38.670623 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.670594    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afe0dd73-a436-4f93-9420-fdd562172910-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-75b8cbf7b5-jmflm\" (UID: \"afe0dd73-a436-4f93-9420-fdd562172910\") " pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:56:38.671811 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.671792    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/afe0dd73-a436-4f93-9420-fdd562172910-secret-metrics-server-client-certs\") pod \"metrics-server-75b8cbf7b5-jmflm\" (UID: \"afe0dd73-a436-4f93-9420-fdd562172910\") " pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:56:38.671886 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.671832    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/afe0dd73-a436-4f93-9420-fdd562172910-secret-metrics-server-tls\") pod \"metrics-server-75b8cbf7b5-jmflm\" (UID: \"afe0dd73-a436-4f93-9420-fdd562172910\") " pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:56:38.671886 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.671842    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/afe0dd73-a436-4f93-9420-fdd562172910-client-ca-bundle\") pod \"metrics-server-75b8cbf7b5-jmflm\" (UID: \"afe0dd73-a436-4f93-9420-fdd562172910\") " pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:56:38.677777 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.677753    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n92tk\" (UniqueName: \"kubernetes.io/projected/afe0dd73-a436-4f93-9420-fdd562172910-kube-api-access-n92tk\") pod \"metrics-server-75b8cbf7b5-jmflm\" (UID: \"afe0dd73-a436-4f93-9420-fdd562172910\") " pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:56:38.822756 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.822663    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:56:38.945236 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:38.945207    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"]
May 28 00:56:38.950465 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:56:38.950435    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafe0dd73_a436_4f93_9420_fdd562172910.slice/crio-1aa5fd7500e44a703e4528560accd93bd5ab8ca5ec363ecab0c26f9f8a641a12 WatchSource:0}: Error finding container 1aa5fd7500e44a703e4528560accd93bd5ab8ca5ec363ecab0c26f9f8a641a12: Status 404 returned error can't find the container with id 1aa5fd7500e44a703e4528560accd93bd5ab8ca5ec363ecab0c26f9f8a641a12
May 28 00:56:39.560302 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:39.560271    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm" event={"ID":"afe0dd73-a436-4f93-9420-fdd562172910","Type":"ContainerStarted","Data":"1aa5fd7500e44a703e4528560accd93bd5ab8ca5ec363ecab0c26f9f8a641a12"}
May 28 00:56:41.436719 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.436686    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f6bdcb5cd-mhp4d"]
May 28 00:56:41.439726 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.439706    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:41.441918 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.441896    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-v9bxw\""
May 28 00:56:41.442261 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.442234    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
May 28 00:56:41.442363 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.442289    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
May 28 00:56:41.442430 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.442390    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
May 28 00:56:41.442490 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.442437    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
May 28 00:56:41.442842 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.442822    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
May 28 00:56:41.442842 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.442840    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
May 28 00:56:41.443002 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.442851    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
May 28 00:56:41.450565 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.450543    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f6bdcb5cd-mhp4d"]
May 28 00:56:41.451174 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.451156    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
May 28 00:56:41.469359 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.469320    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9z98m"
May 28 00:56:41.494678 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.494657    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-trusted-ca-bundle\") pod \"console-7f6bdcb5cd-mhp4d\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") " pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:41.494794 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.494691    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2r4w\" (UniqueName: \"kubernetes.io/projected/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-kube-api-access-p2r4w\") pod \"console-7f6bdcb5cd-mhp4d\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") " pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:41.494794 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.494712    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-service-ca\") pod \"console-7f6bdcb5cd-mhp4d\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") " pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:41.494794 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.494775    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-console-config\") pod \"console-7f6bdcb5cd-mhp4d\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") " pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:41.494794 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.494791    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-oauth-serving-cert\") pod \"console-7f6bdcb5cd-mhp4d\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") " pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:41.494967 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.494841    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-console-serving-cert\") pod \"console-7f6bdcb5cd-mhp4d\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") " pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:41.494967 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.494897    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-console-oauth-config\") pod \"console-7f6bdcb5cd-mhp4d\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") " pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:41.567728 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.567695    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm" event={"ID":"afe0dd73-a436-4f93-9420-fdd562172910","Type":"ContainerStarted","Data":"37645e62d0badfa318d4ac0cbc75e2b2dcbd12ae900faece3fd03024cf875e75"}
May 28 00:56:41.570637 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.570606    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d0f24db-4a2d-4454-a9c2-5d87a855f953","Type":"ContainerStarted","Data":"08694e6a7213d0c736d8705e416df6c46f0a8c2886df15cf6ec6994be3c6abab"}
May 28 00:56:41.570637 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.570639    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d0f24db-4a2d-4454-a9c2-5d87a855f953","Type":"ContainerStarted","Data":"fb041bccad2b0747200de7f3701f6951d4094f61f05b5c8d5cfad63a506c7411"}
May 28 00:56:41.570829 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.570653    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d0f24db-4a2d-4454-a9c2-5d87a855f953","Type":"ContainerStarted","Data":"6a150a0d29e9773ad0569f0ff26f8f2d5f20fc3789fff635e441258b7b9d3f6d"}
May 28 00:56:41.570829 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.570667    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d0f24db-4a2d-4454-a9c2-5d87a855f953","Type":"ContainerStarted","Data":"038427d942c06b56c779c54973a238cdfe41b4acfe97c72c9bf0f02673f45ec1"}
May 28 00:56:41.570829 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.570678    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d0f24db-4a2d-4454-a9c2-5d87a855f953","Type":"ContainerStarted","Data":"93a79c73015262201ae9eb4fac3e6c70ec11303357518a986e07127dadb70744"}
May 28 00:56:41.570829 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.570688    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d0f24db-4a2d-4454-a9c2-5d87a855f953","Type":"ContainerStarted","Data":"a2d93e2bd2928c65dbc9321aa428ad1c00e64727931ef48363a45234f8625abf"}
May 28 00:56:41.588685 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.588630    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm" podStartSLOduration=1.96670253 podStartE2EDuration="3.588613475s" podCreationTimestamp="2026-05-28 00:56:38 +0000 UTC" firstStartedPulling="2026-05-28 00:56:38.952863024 +0000 UTC m=+58.233820096" lastFinishedPulling="2026-05-28 00:56:40.574773969 +0000 UTC m=+59.855731041" observedRunningTime="2026-05-28 00:56:41.587265907 +0000 UTC m=+60.868222990" watchObservedRunningTime="2026-05-28 00:56:41.588613475 +0000 UTC m=+60.869570568"
May 28 00:56:41.595925 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.595900    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-console-config\") pod \"console-7f6bdcb5cd-mhp4d\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") " pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:41.596020 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.595932    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-oauth-serving-cert\") pod \"console-7f6bdcb5cd-mhp4d\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") " pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:41.596020 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.595953    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-console-serving-cert\") pod \"console-7f6bdcb5cd-mhp4d\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") " pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:41.596020 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.595986    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-console-oauth-config\") pod \"console-7f6bdcb5cd-mhp4d\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") " pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:41.596217 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.596187    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-trusted-ca-bundle\") pod \"console-7f6bdcb5cd-mhp4d\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") " pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:41.596277 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.596258    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p2r4w\" (UniqueName: \"kubernetes.io/projected/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-kube-api-access-p2r4w\") pod \"console-7f6bdcb5cd-mhp4d\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") " pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:41.596328 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.596307    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-service-ca\") pod \"console-7f6bdcb5cd-mhp4d\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") " pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:41.596737 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.596708    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-oauth-serving-cert\") pod \"console-7f6bdcb5cd-mhp4d\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") " pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:41.596737 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.596725    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-console-config\") pod \"console-7f6bdcb5cd-mhp4d\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") " pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:41.597303 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.597282    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-service-ca\") pod \"console-7f6bdcb5cd-mhp4d\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") " pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:41.597650 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.597631    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-trusted-ca-bundle\") pod \"console-7f6bdcb5cd-mhp4d\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") " pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:41.598614 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.598589    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-console-oauth-config\") pod \"console-7f6bdcb5cd-mhp4d\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") " pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:41.598871 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.598849    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-console-serving-cert\") pod \"console-7f6bdcb5cd-mhp4d\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") " pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:41.603881 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.603858    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2r4w\" (UniqueName: \"kubernetes.io/projected/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-kube-api-access-p2r4w\") pod \"console-7f6bdcb5cd-mhp4d\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") " pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:41.621047 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.620996    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.285537471 podStartE2EDuration="6.620979235s" podCreationTimestamp="2026-05-28 00:56:35 +0000 UTC" firstStartedPulling="2026-05-28 00:56:36.158921776 +0000 UTC m=+55.439878848" lastFinishedPulling="2026-05-28 00:56:41.494363538 +0000 UTC m=+60.775320612" observedRunningTime="2026-05-28 00:56:41.618088717 +0000 UTC m=+60.899045810" watchObservedRunningTime="2026-05-28 00:56:41.620979235 +0000 UTC m=+60.901936329"
May 28 00:56:41.753034 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.752946    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:41.883512 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:41.883313    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f6bdcb5cd-mhp4d"]
May 28 00:56:41.886006 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:56:41.885979    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3d47ac1_5bf7_4da1_831f_01b1b1ef55e3.slice/crio-8f2fcb6e7c7b7dc995db4cb98a98544b7bb727c9e892088ed5e827abfb4bbbb2 WatchSource:0}: Error finding container 8f2fcb6e7c7b7dc995db4cb98a98544b7bb727c9e892088ed5e827abfb4bbbb2: Status 404 returned error can't find the container with id 8f2fcb6e7c7b7dc995db4cb98a98544b7bb727c9e892088ed5e827abfb4bbbb2
May 28 00:56:42.574442 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:42.574392    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f6bdcb5cd-mhp4d" event={"ID":"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3","Type":"ContainerStarted","Data":"8f2fcb6e7c7b7dc995db4cb98a98544b7bb727c9e892088ed5e827abfb4bbbb2"}
May 28 00:56:44.543370 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:44.543337    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-57vdg"
May 28 00:56:44.585694 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:44.585610    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f6bdcb5cd-mhp4d" event={"ID":"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3","Type":"ContainerStarted","Data":"449689a6008b562aeceb9f68d17215377cf91cbec41f097898b6ae836848f086"}
May 28 00:56:44.601990 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:44.601813    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-7f6bdcb5cd-mhp4d" podStartSLOduration=1.157294475 podStartE2EDuration="3.601795987s" podCreationTimestamp="2026-05-28 00:56:41 +0000 UTC" firstStartedPulling="2026-05-28 00:56:41.887908195 +0000 UTC m=+61.168865265" lastFinishedPulling="2026-05-28 00:56:44.332409698 +0000 UTC m=+63.613366777" observedRunningTime="2026-05-28 00:56:44.601379859 +0000 UTC m=+63.882336956" watchObservedRunningTime="2026-05-28 00:56:44.601795987 +0000 UTC m=+63.882753081"
May 28 00:56:47.047726 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:47.047684    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d04c1e7-069a-413e-a9f6-d87a8095844d-metrics-certs\") pod \"network-metrics-daemon-f8h8b\" (UID: \"3d04c1e7-069a-413e-a9f6-d87a8095844d\") " pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:56:47.050032 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:47.050007    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 00:56:47.060759 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:47.060736    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d04c1e7-069a-413e-a9f6-d87a8095844d-metrics-certs\") pod \"network-metrics-daemon-f8h8b\" (UID: \"3d04c1e7-069a-413e-a9f6-d87a8095844d\") " pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:56:47.131266 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:47.131240    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-x6qxc\""
May 28 00:56:47.139478 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:47.139461    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f8h8b"
May 28 00:56:47.249305 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:47.249274    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fzwm8\" (UniqueName: \"kubernetes.io/projected/6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7-kube-api-access-fzwm8\") pod \"network-check-target-l9q4k\" (UID: \"6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7\") " pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:56:47.251697 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:47.251668    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 00:56:47.262821 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:47.262798    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 00:56:47.273384 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:47.273360    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzwm8\" (UniqueName: \"kubernetes.io/projected/6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7-kube-api-access-fzwm8\") pod \"network-check-target-l9q4k\" (UID: \"6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7\") " pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:56:47.273548 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:47.273529    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-f8h8b"]
May 28 00:56:47.276487 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:56:47.276463    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d04c1e7_069a_413e_a9f6_d87a8095844d.slice/crio-8e310c42caab2240dd3e67830fbab386f6fabac4b10124d53a410056537b4f19 WatchSource:0}: Error finding container 8e310c42caab2240dd3e67830fbab386f6fabac4b10124d53a410056537b4f19: Status 404 returned error can't find the container with id 8e310c42caab2240dd3e67830fbab386f6fabac4b10124d53a410056537b4f19
May 28 00:56:47.442241 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:47.442212    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g7dls\""
May 28 00:56:47.450976 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:47.450954    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:56:47.573102 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:47.573068    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l9q4k"]
May 28 00:56:47.576544 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:56:47.576520    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fa8fafd_34d7_4feb_8a22_7b1f3fcba8a7.slice/crio-26d26d4fd88e1749414bc535449270850ec3811f6dd34e77282e7e17a43f0364 WatchSource:0}: Error finding container 26d26d4fd88e1749414bc535449270850ec3811f6dd34e77282e7e17a43f0364: Status 404 returned error can't find the container with id 26d26d4fd88e1749414bc535449270850ec3811f6dd34e77282e7e17a43f0364
May 28 00:56:47.593353 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:47.593306    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l9q4k" event={"ID":"6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7","Type":"ContainerStarted","Data":"26d26d4fd88e1749414bc535449270850ec3811f6dd34e77282e7e17a43f0364"}
May 28 00:56:47.594264 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:47.594243    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f8h8b" event={"ID":"3d04c1e7-069a-413e-a9f6-d87a8095844d","Type":"ContainerStarted","Data":"8e310c42caab2240dd3e67830fbab386f6fabac4b10124d53a410056537b4f19"}
May 28 00:56:49.605048 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:49.605010    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f8h8b" event={"ID":"3d04c1e7-069a-413e-a9f6-d87a8095844d","Type":"ContainerStarted","Data":"9f7e941292a4b800c02fb5bf0671c102bef93f5745156f66d81c3ccc6059e6d5"}
May 28 00:56:49.605048 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:49.605048    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f8h8b" event={"ID":"3d04c1e7-069a-413e-a9f6-d87a8095844d","Type":"ContainerStarted","Data":"0d9f6bf62173b2ec7b767526d471f614e86f2ae493fdf45e95676197541b95ee"}
May 28 00:56:49.623347 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:49.623114    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-f8h8b" podStartSLOduration=67.336796541 podStartE2EDuration="1m8.623096148s" podCreationTimestamp="2026-05-28 00:55:41 +0000 UTC" firstStartedPulling="2026-05-28 00:56:47.278644874 +0000 UTC m=+66.559601949" lastFinishedPulling="2026-05-28 00:56:48.564944468 +0000 UTC m=+67.845901556" observedRunningTime="2026-05-28 00:56:49.622881693 +0000 UTC m=+68.903838807" watchObservedRunningTime="2026-05-28 00:56:49.623096148 +0000 UTC m=+68.904053243"
May 28 00:56:51.611908 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:51.611870    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l9q4k" event={"ID":"6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7","Type":"ContainerStarted","Data":"c0727229e2fce6a43213c9e37674c7f5f1816c9fbb5a6bf5b8e7c14fd89f3c3b"}
May 28 00:56:51.612327 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:51.612045    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:56:51.628107 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:51.628052    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-l9q4k" podStartSLOduration=67.547918971 podStartE2EDuration="1m10.628036703s" podCreationTimestamp="2026-05-28 00:55:41 +0000 UTC" firstStartedPulling="2026-05-28 00:56:47.578334031 +0000 UTC m=+66.859291102" lastFinishedPulling="2026-05-28 00:56:50.658451763 +0000 UTC m=+69.939408834" observedRunningTime="2026-05-28 00:56:51.626749746 +0000 UTC m=+70.907706838" watchObservedRunningTime="2026-05-28 00:56:51.628036703 +0000 UTC m=+70.908993795"
May 28 00:56:51.753535 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:51.753487    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:51.753535 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:51.753539    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:51.758310 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:51.758287    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:52.521993 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:52.521887    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5997b765f9-fcpb7"
May 28 00:56:52.618860 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:52.618833    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:56:58.823113 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:58.823081    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:56:58.823575 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:56:58.823157    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:57:01.369612 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:01.369576    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42201706-6a2e-412d-8072-15c3476baba9-original-pull-secret\") pod \"global-pull-secret-syncer-qw4tp\" (UID: \"42201706-6a2e-412d-8072-15c3476baba9\") " pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:57:01.372005 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:01.371987    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 28 00:57:01.381861 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:01.381836    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42201706-6a2e-412d-8072-15c3476baba9-original-pull-secret\") pod \"global-pull-secret-syncer-qw4tp\" (UID: \"42201706-6a2e-412d-8072-15c3476baba9\") " pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:57:01.535241 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:01.535207    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qw4tp"
May 28 00:57:01.678544 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:01.678504    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qw4tp"]
May 28 00:57:01.682359 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:57:01.682333    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42201706_6a2e_412d_8072_15c3476baba9.slice/crio-b21e24b9008a6d150fec016e682697721dd20c5580874f7439e8d0796f4e0f75 WatchSource:0}: Error finding container b21e24b9008a6d150fec016e682697721dd20c5580874f7439e8d0796f4e0f75: Status 404 returned error can't find the container with id b21e24b9008a6d150fec016e682697721dd20c5580874f7439e8d0796f4e0f75
May 28 00:57:02.644531 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:02.644494    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qw4tp" event={"ID":"42201706-6a2e-412d-8072-15c3476baba9","Type":"ContainerStarted","Data":"b21e24b9008a6d150fec016e682697721dd20c5580874f7439e8d0796f4e0f75"}
May 28 00:57:07.659733 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:07.659693    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qw4tp" event={"ID":"42201706-6a2e-412d-8072-15c3476baba9","Type":"ContainerStarted","Data":"6c06beaa3ca551b2383309919f972cfdf459bdbb9518afd14a517c273854b6b6"}
May 28 00:57:07.676016 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:07.675972    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qw4tp" podStartSLOduration=65.704605191 podStartE2EDuration="1m10.675957163s" podCreationTimestamp="2026-05-28 00:55:57 +0000 UTC" firstStartedPulling="2026-05-28 00:57:01.684423685 +0000 UTC m=+80.965380769" lastFinishedPulling="2026-05-28 00:57:06.655775669 +0000 UTC m=+85.936732741" observedRunningTime="2026-05-28 00:57:07.674584123 +0000 UTC m=+86.955541213" watchObservedRunningTime="2026-05-28 00:57:07.675957163 +0000 UTC m=+86.956914256"
May 28 00:57:13.739554 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:13.739522    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gbczp_52d52ce4-3bef-48d9-af17-cd6d77829482/serve-healthcheck-canary/0.log"
May 28 00:57:18.829057 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:18.829029    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:57:18.833032 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:18.833000    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-75b8cbf7b5-jmflm"
May 28 00:57:22.617280 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:22.617248    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-l9q4k"
May 28 00:57:54.265822 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:54.265788    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 00:57:54.266399 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:54.266336    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerName="alertmanager" containerID="cri-o://a2d93e2bd2928c65dbc9321aa428ad1c00e64727931ef48363a45234f8625abf" gracePeriod=120
May 28 00:57:54.266529 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:54.266406    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerName="prom-label-proxy" containerID="cri-o://08694e6a7213d0c736d8705e416df6c46f0a8c2886df15cf6ec6994be3c6abab" gracePeriod=120
May 28 00:57:54.266529 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:54.266447    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerName="config-reloader" containerID="cri-o://93a79c73015262201ae9eb4fac3e6c70ec11303357518a986e07127dadb70744" gracePeriod=120
May 28 00:57:54.266529 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:54.266475    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerName="kube-rbac-proxy" containerID="cri-o://6a150a0d29e9773ad0569f0ff26f8f2d5f20fc3789fff635e441258b7b9d3f6d" gracePeriod=120
May 28 00:57:54.266529 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:54.266394    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerName="kube-rbac-proxy-metric" containerID="cri-o://fb041bccad2b0747200de7f3701f6951d4094f61f05b5c8d5cfad63a506c7411" gracePeriod=120
May 28 00:57:54.266797 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:54.266735    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerName="kube-rbac-proxy-web" containerID="cri-o://038427d942c06b56c779c54973a238cdfe41b4acfe97c72c9bf0f02673f45ec1" gracePeriod=120
May 28 00:57:54.802552 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:54.802512    2550 generic.go:358] "Generic (PLEG): container finished" podID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerID="08694e6a7213d0c736d8705e416df6c46f0a8c2886df15cf6ec6994be3c6abab" exitCode=0
May 28 00:57:54.802552 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:54.802539    2550 generic.go:358] "Generic (PLEG): container finished" podID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerID="6a150a0d29e9773ad0569f0ff26f8f2d5f20fc3789fff635e441258b7b9d3f6d" exitCode=0
May 28 00:57:54.802552 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:54.802545    2550 generic.go:358] "Generic (PLEG): container finished" podID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerID="93a79c73015262201ae9eb4fac3e6c70ec11303357518a986e07127dadb70744" exitCode=0
May 28 00:57:54.802552 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:54.802550    2550 generic.go:358] "Generic (PLEG): container finished" podID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerID="a2d93e2bd2928c65dbc9321aa428ad1c00e64727931ef48363a45234f8625abf" exitCode=0
May 28 00:57:54.802821 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:54.802583    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d0f24db-4a2d-4454-a9c2-5d87a855f953","Type":"ContainerDied","Data":"08694e6a7213d0c736d8705e416df6c46f0a8c2886df15cf6ec6994be3c6abab"}
May 28 00:57:54.802821 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:54.802616    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d0f24db-4a2d-4454-a9c2-5d87a855f953","Type":"ContainerDied","Data":"6a150a0d29e9773ad0569f0ff26f8f2d5f20fc3789fff635e441258b7b9d3f6d"}
May 28 00:57:54.802821 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:54.802626    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d0f24db-4a2d-4454-a9c2-5d87a855f953","Type":"ContainerDied","Data":"93a79c73015262201ae9eb4fac3e6c70ec11303357518a986e07127dadb70744"}
May 28 00:57:54.802821 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:54.802635    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d0f24db-4a2d-4454-a9c2-5d87a855f953","Type":"ContainerDied","Data":"a2d93e2bd2928c65dbc9321aa428ad1c00e64727931ef48363a45234f8625abf"}
May 28 00:57:55.511742 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.511710    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:55.558992 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.558951    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d0f24db-4a2d-4454-a9c2-5d87a855f953-tls-assets\") pod \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") "
May 28 00:57:55.559166 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.559005    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d0f24db-4a2d-4454-a9c2-5d87a855f953-metrics-client-ca\") pod \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") "
May 28 00:57:55.559166 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.559067    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-secret-alertmanager-kube-rbac-proxy\") pod \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") "
May 28 00:57:55.559166 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.559085    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-cluster-tls-config\") pod \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") "
May 28 00:57:55.559166 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.559108    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-web-config\") pod \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") "
May 28 00:57:55.559166 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.559123    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-config-volume\") pod \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") "
May 28 00:57:55.559166 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.559148    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-secret-alertmanager-main-tls\") pod \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") "
May 28 00:57:55.559515 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.559183    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2d0f24db-4a2d-4454-a9c2-5d87a855f953-alertmanager-main-db\") pod \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") "
May 28 00:57:55.559515 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.559215    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-secret-alertmanager-kube-rbac-proxy-metric\") pod \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") "
May 28 00:57:55.559515 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.559271    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d0f24db-4a2d-4454-a9c2-5d87a855f953-alertmanager-trusted-ca-bundle\") pod \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") "
May 28 00:57:55.559515 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.559301    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d0f24db-4a2d-4454-a9c2-5d87a855f953-config-out\") pod \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") "
May 28 00:57:55.559515 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.559325    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-576jz\" (UniqueName: \"kubernetes.io/projected/2d0f24db-4a2d-4454-a9c2-5d87a855f953-kube-api-access-576jz\") pod \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") "
May 28 00:57:55.559515 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.559356    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-secret-alertmanager-kube-rbac-proxy-web\") pod \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\" (UID: \"2d0f24db-4a2d-4454-a9c2-5d87a855f953\") "
May 28 00:57:55.559515 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.559441    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d0f24db-4a2d-4454-a9c2-5d87a855f953-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "2d0f24db-4a2d-4454-a9c2-5d87a855f953" (UID: "2d0f24db-4a2d-4454-a9c2-5d87a855f953"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 00:57:55.559849 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.559648    2550 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d0f24db-4a2d-4454-a9c2-5d87a855f953-metrics-client-ca\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:57:55.560661 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.560439    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d0f24db-4a2d-4454-a9c2-5d87a855f953-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "2d0f24db-4a2d-4454-a9c2-5d87a855f953" (UID: "2d0f24db-4a2d-4454-a9c2-5d87a855f953"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 00:57:55.560661 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.560476    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d0f24db-4a2d-4454-a9c2-5d87a855f953-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "2d0f24db-4a2d-4454-a9c2-5d87a855f953" (UID: "2d0f24db-4a2d-4454-a9c2-5d87a855f953"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 00:57:55.562686 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.562604    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "2d0f24db-4a2d-4454-a9c2-5d87a855f953" (UID: "2d0f24db-4a2d-4454-a9c2-5d87a855f953"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 00:57:55.563092 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.563047    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d0f24db-4a2d-4454-a9c2-5d87a855f953-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "2d0f24db-4a2d-4454-a9c2-5d87a855f953" (UID: "2d0f24db-4a2d-4454-a9c2-5d87a855f953"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 00:57:55.563692 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.563645    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "2d0f24db-4a2d-4454-a9c2-5d87a855f953" (UID: "2d0f24db-4a2d-4454-a9c2-5d87a855f953"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 00:57:55.564199 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.564158    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "2d0f24db-4a2d-4454-a9c2-5d87a855f953" (UID: "2d0f24db-4a2d-4454-a9c2-5d87a855f953"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 00:57:55.564682 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.564655    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d0f24db-4a2d-4454-a9c2-5d87a855f953-config-out" (OuterVolumeSpecName: "config-out") pod "2d0f24db-4a2d-4454-a9c2-5d87a855f953" (UID: "2d0f24db-4a2d-4454-a9c2-5d87a855f953"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 00:57:55.564768 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.564655    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "2d0f24db-4a2d-4454-a9c2-5d87a855f953" (UID: "2d0f24db-4a2d-4454-a9c2-5d87a855f953"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 00:57:55.565050 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.565021    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-config-volume" (OuterVolumeSpecName: "config-volume") pod "2d0f24db-4a2d-4454-a9c2-5d87a855f953" (UID: "2d0f24db-4a2d-4454-a9c2-5d87a855f953"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 00:57:55.565329 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.565305    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d0f24db-4a2d-4454-a9c2-5d87a855f953-kube-api-access-576jz" (OuterVolumeSpecName: "kube-api-access-576jz") pod "2d0f24db-4a2d-4454-a9c2-5d87a855f953" (UID: "2d0f24db-4a2d-4454-a9c2-5d87a855f953"). InnerVolumeSpecName "kube-api-access-576jz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 00:57:55.567689 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.567648    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "2d0f24db-4a2d-4454-a9c2-5d87a855f953" (UID: "2d0f24db-4a2d-4454-a9c2-5d87a855f953"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 00:57:55.572915 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.572889    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-web-config" (OuterVolumeSpecName: "web-config") pod "2d0f24db-4a2d-4454-a9c2-5d87a855f953" (UID: "2d0f24db-4a2d-4454-a9c2-5d87a855f953"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 00:57:55.660253 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.660198    2550 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:57:55.660253 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.660243    2550 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d0f24db-4a2d-4454-a9c2-5d87a855f953-tls-assets\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:57:55.660253 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.660255    2550 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:57:55.660253 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.660265    2550 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-cluster-tls-config\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:57:55.660253 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.660275    2550 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-web-config\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:57:55.660595 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.660284    2550 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-config-volume\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:57:55.660595 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.660293    2550 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-secret-alertmanager-main-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:57:55.660595 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.660302    2550 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2d0f24db-4a2d-4454-a9c2-5d87a855f953-alertmanager-main-db\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:57:55.660595 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.660310    2550 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2d0f24db-4a2d-4454-a9c2-5d87a855f953-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:57:55.660595 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.660319    2550 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d0f24db-4a2d-4454-a9c2-5d87a855f953-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:57:55.660595 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.660328    2550 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d0f24db-4a2d-4454-a9c2-5d87a855f953-config-out\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:57:55.660595 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.660337    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-576jz\" (UniqueName: \"kubernetes.io/projected/2d0f24db-4a2d-4454-a9c2-5d87a855f953-kube-api-access-576jz\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:57:55.808368 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.808331    2550 generic.go:358] "Generic (PLEG): container finished" podID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerID="fb041bccad2b0747200de7f3701f6951d4094f61f05b5c8d5cfad63a506c7411" exitCode=0
May 28 00:57:55.808368 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.808358    2550 generic.go:358] "Generic (PLEG): container finished" podID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerID="038427d942c06b56c779c54973a238cdfe41b4acfe97c72c9bf0f02673f45ec1" exitCode=0
May 28 00:57:55.808638 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.808392    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d0f24db-4a2d-4454-a9c2-5d87a855f953","Type":"ContainerDied","Data":"fb041bccad2b0747200de7f3701f6951d4094f61f05b5c8d5cfad63a506c7411"}
May 28 00:57:55.808638 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.808436    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d0f24db-4a2d-4454-a9c2-5d87a855f953","Type":"ContainerDied","Data":"038427d942c06b56c779c54973a238cdfe41b4acfe97c72c9bf0f02673f45ec1"}
May 28 00:57:55.808638 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.808453    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d0f24db-4a2d-4454-a9c2-5d87a855f953","Type":"ContainerDied","Data":"8d30c7d4a7a440c7c2fe02347c835db2af8ef5063591b876fbe3dc2d49500d48"}
May 28 00:57:55.808638 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.808488    2550 scope.go:117] "RemoveContainer" containerID="08694e6a7213d0c736d8705e416df6c46f0a8c2886df15cf6ec6994be3c6abab"
May 28 00:57:55.808638 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.808519    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:55.816138 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.816120    2550 scope.go:117] "RemoveContainer" containerID="fb041bccad2b0747200de7f3701f6951d4094f61f05b5c8d5cfad63a506c7411"
May 28 00:57:55.822950 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.822928    2550 scope.go:117] "RemoveContainer" containerID="6a150a0d29e9773ad0569f0ff26f8f2d5f20fc3789fff635e441258b7b9d3f6d"
May 28 00:57:55.829545 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.829524    2550 scope.go:117] "RemoveContainer" containerID="038427d942c06b56c779c54973a238cdfe41b4acfe97c72c9bf0f02673f45ec1"
May 28 00:57:55.831510 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.831482    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 00:57:55.835766 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.835739    2550 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 00:57:55.837221 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.837206    2550 scope.go:117] "RemoveContainer" containerID="93a79c73015262201ae9eb4fac3e6c70ec11303357518a986e07127dadb70744"
May 28 00:57:55.843980 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.843963    2550 scope.go:117] "RemoveContainer" containerID="a2d93e2bd2928c65dbc9321aa428ad1c00e64727931ef48363a45234f8625abf"
May 28 00:57:55.850444 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.850397    2550 scope.go:117] "RemoveContainer" containerID="981d77e03484b01e7da205fea04ce8ed7ec62503b70687d28875ab003e1c047c"
May 28 00:57:55.856759 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.856740    2550 scope.go:117] "RemoveContainer" containerID="08694e6a7213d0c736d8705e416df6c46f0a8c2886df15cf6ec6994be3c6abab"
May 28 00:57:55.857040 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:57:55.857011    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08694e6a7213d0c736d8705e416df6c46f0a8c2886df15cf6ec6994be3c6abab\": container with ID starting with 08694e6a7213d0c736d8705e416df6c46f0a8c2886df15cf6ec6994be3c6abab not found: ID does not exist" containerID="08694e6a7213d0c736d8705e416df6c46f0a8c2886df15cf6ec6994be3c6abab"
May 28 00:57:55.857102 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.857050    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08694e6a7213d0c736d8705e416df6c46f0a8c2886df15cf6ec6994be3c6abab"} err="failed to get container status \"08694e6a7213d0c736d8705e416df6c46f0a8c2886df15cf6ec6994be3c6abab\": rpc error: code = NotFound desc = could not find container \"08694e6a7213d0c736d8705e416df6c46f0a8c2886df15cf6ec6994be3c6abab\": container with ID starting with 08694e6a7213d0c736d8705e416df6c46f0a8c2886df15cf6ec6994be3c6abab not found: ID does not exist"
May 28 00:57:55.857102 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.857087    2550 scope.go:117] "RemoveContainer" containerID="fb041bccad2b0747200de7f3701f6951d4094f61f05b5c8d5cfad63a506c7411"
May 28 00:57:55.857351 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:57:55.857333    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb041bccad2b0747200de7f3701f6951d4094f61f05b5c8d5cfad63a506c7411\": container with ID starting with fb041bccad2b0747200de7f3701f6951d4094f61f05b5c8d5cfad63a506c7411 not found: ID does not exist" containerID="fb041bccad2b0747200de7f3701f6951d4094f61f05b5c8d5cfad63a506c7411"
May 28 00:57:55.857393 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.857357    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb041bccad2b0747200de7f3701f6951d4094f61f05b5c8d5cfad63a506c7411"} err="failed to get container status \"fb041bccad2b0747200de7f3701f6951d4094f61f05b5c8d5cfad63a506c7411\": rpc error: code = NotFound desc = could not find container \"fb041bccad2b0747200de7f3701f6951d4094f61f05b5c8d5cfad63a506c7411\": container with ID starting with fb041bccad2b0747200de7f3701f6951d4094f61f05b5c8d5cfad63a506c7411 not found: ID does not exist"
May 28 00:57:55.857393 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.857377    2550 scope.go:117] "RemoveContainer" containerID="6a150a0d29e9773ad0569f0ff26f8f2d5f20fc3789fff635e441258b7b9d3f6d"
May 28 00:57:55.857640 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:57:55.857623    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a150a0d29e9773ad0569f0ff26f8f2d5f20fc3789fff635e441258b7b9d3f6d\": container with ID starting with 6a150a0d29e9773ad0569f0ff26f8f2d5f20fc3789fff635e441258b7b9d3f6d not found: ID does not exist" containerID="6a150a0d29e9773ad0569f0ff26f8f2d5f20fc3789fff635e441258b7b9d3f6d"
May 28 00:57:55.857679 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.857643    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a150a0d29e9773ad0569f0ff26f8f2d5f20fc3789fff635e441258b7b9d3f6d"} err="failed to get container status \"6a150a0d29e9773ad0569f0ff26f8f2d5f20fc3789fff635e441258b7b9d3f6d\": rpc error: code = NotFound desc = could not find container \"6a150a0d29e9773ad0569f0ff26f8f2d5f20fc3789fff635e441258b7b9d3f6d\": container with ID starting with 6a150a0d29e9773ad0569f0ff26f8f2d5f20fc3789fff635e441258b7b9d3f6d not found: ID does not exist"
May 28 00:57:55.857679 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.857657    2550 scope.go:117] "RemoveContainer" containerID="038427d942c06b56c779c54973a238cdfe41b4acfe97c72c9bf0f02673f45ec1"
May 28 00:57:55.857870 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:57:55.857854    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"038427d942c06b56c779c54973a238cdfe41b4acfe97c72c9bf0f02673f45ec1\": container with ID starting with 038427d942c06b56c779c54973a238cdfe41b4acfe97c72c9bf0f02673f45ec1 not found: ID does not exist" containerID="038427d942c06b56c779c54973a238cdfe41b4acfe97c72c9bf0f02673f45ec1"
May 28 00:57:55.857909 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.857873    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"038427d942c06b56c779c54973a238cdfe41b4acfe97c72c9bf0f02673f45ec1"} err="failed to get container status \"038427d942c06b56c779c54973a238cdfe41b4acfe97c72c9bf0f02673f45ec1\": rpc error: code = NotFound desc = could not find container \"038427d942c06b56c779c54973a238cdfe41b4acfe97c72c9bf0f02673f45ec1\": container with ID starting with 038427d942c06b56c779c54973a238cdfe41b4acfe97c72c9bf0f02673f45ec1 not found: ID does not exist"
May 28 00:57:55.857909 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.857886    2550 scope.go:117] "RemoveContainer" containerID="93a79c73015262201ae9eb4fac3e6c70ec11303357518a986e07127dadb70744"
May 28 00:57:55.858117 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:57:55.858101    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93a79c73015262201ae9eb4fac3e6c70ec11303357518a986e07127dadb70744\": container with ID starting with 93a79c73015262201ae9eb4fac3e6c70ec11303357518a986e07127dadb70744 not found: ID does not exist" containerID="93a79c73015262201ae9eb4fac3e6c70ec11303357518a986e07127dadb70744"
May 28 00:57:55.858157 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.858123    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93a79c73015262201ae9eb4fac3e6c70ec11303357518a986e07127dadb70744"} err="failed to get container status \"93a79c73015262201ae9eb4fac3e6c70ec11303357518a986e07127dadb70744\": rpc error: code = NotFound desc = could not find container \"93a79c73015262201ae9eb4fac3e6c70ec11303357518a986e07127dadb70744\": container with ID starting with 93a79c73015262201ae9eb4fac3e6c70ec11303357518a986e07127dadb70744 not found: ID does not exist"
May 28 00:57:55.858157 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.858138    2550 scope.go:117] "RemoveContainer" containerID="a2d93e2bd2928c65dbc9321aa428ad1c00e64727931ef48363a45234f8625abf"
May 28 00:57:55.858354 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:57:55.858332    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2d93e2bd2928c65dbc9321aa428ad1c00e64727931ef48363a45234f8625abf\": container with ID starting with a2d93e2bd2928c65dbc9321aa428ad1c00e64727931ef48363a45234f8625abf not found: ID does not exist" containerID="a2d93e2bd2928c65dbc9321aa428ad1c00e64727931ef48363a45234f8625abf"
May 28 00:57:55.858399 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.858352    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2d93e2bd2928c65dbc9321aa428ad1c00e64727931ef48363a45234f8625abf"} err="failed to get container status \"a2d93e2bd2928c65dbc9321aa428ad1c00e64727931ef48363a45234f8625abf\": rpc error: code = NotFound desc = could not find container \"a2d93e2bd2928c65dbc9321aa428ad1c00e64727931ef48363a45234f8625abf\": container with ID starting with a2d93e2bd2928c65dbc9321aa428ad1c00e64727931ef48363a45234f8625abf not found: ID does not exist"
May 28 00:57:55.858399 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.858364    2550 scope.go:117] "RemoveContainer" containerID="981d77e03484b01e7da205fea04ce8ed7ec62503b70687d28875ab003e1c047c"
May 28 00:57:55.858680 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:57:55.858662    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"981d77e03484b01e7da205fea04ce8ed7ec62503b70687d28875ab003e1c047c\": container with ID starting with 981d77e03484b01e7da205fea04ce8ed7ec62503b70687d28875ab003e1c047c not found: ID does not exist" containerID="981d77e03484b01e7da205fea04ce8ed7ec62503b70687d28875ab003e1c047c"
May 28 00:57:55.858746 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.858685    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"981d77e03484b01e7da205fea04ce8ed7ec62503b70687d28875ab003e1c047c"} err="failed to get container status \"981d77e03484b01e7da205fea04ce8ed7ec62503b70687d28875ab003e1c047c\": rpc error: code = NotFound desc = could not find container \"981d77e03484b01e7da205fea04ce8ed7ec62503b70687d28875ab003e1c047c\": container with ID starting with 981d77e03484b01e7da205fea04ce8ed7ec62503b70687d28875ab003e1c047c not found: ID does not exist"
May 28 00:57:55.858746 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.858699    2550 scope.go:117] "RemoveContainer" containerID="08694e6a7213d0c736d8705e416df6c46f0a8c2886df15cf6ec6994be3c6abab"
May 28 00:57:55.858890 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.858872    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08694e6a7213d0c736d8705e416df6c46f0a8c2886df15cf6ec6994be3c6abab"} err="failed to get container status \"08694e6a7213d0c736d8705e416df6c46f0a8c2886df15cf6ec6994be3c6abab\": rpc error: code = NotFound desc = could not find container \"08694e6a7213d0c736d8705e416df6c46f0a8c2886df15cf6ec6994be3c6abab\": container with ID starting with 08694e6a7213d0c736d8705e416df6c46f0a8c2886df15cf6ec6994be3c6abab not found: ID does not exist"
May 28 00:57:55.858925 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.858891    2550 scope.go:117] "RemoveContainer" containerID="fb041bccad2b0747200de7f3701f6951d4094f61f05b5c8d5cfad63a506c7411"
May 28 00:57:55.859089 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.859073    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb041bccad2b0747200de7f3701f6951d4094f61f05b5c8d5cfad63a506c7411"} err="failed to get container status \"fb041bccad2b0747200de7f3701f6951d4094f61f05b5c8d5cfad63a506c7411\": rpc error: code = NotFound desc = could not find container \"fb041bccad2b0747200de7f3701f6951d4094f61f05b5c8d5cfad63a506c7411\": container with ID starting with fb041bccad2b0747200de7f3701f6951d4094f61f05b5c8d5cfad63a506c7411 not found: ID does not exist"
May 28 00:57:55.859132 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.859089    2550 scope.go:117] "RemoveContainer" containerID="6a150a0d29e9773ad0569f0ff26f8f2d5f20fc3789fff635e441258b7b9d3f6d"
May 28 00:57:55.859321 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.859304    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a150a0d29e9773ad0569f0ff26f8f2d5f20fc3789fff635e441258b7b9d3f6d"} err="failed to get container status \"6a150a0d29e9773ad0569f0ff26f8f2d5f20fc3789fff635e441258b7b9d3f6d\": rpc error: code = NotFound desc = could not find container \"6a150a0d29e9773ad0569f0ff26f8f2d5f20fc3789fff635e441258b7b9d3f6d\": container with ID starting with 6a150a0d29e9773ad0569f0ff26f8f2d5f20fc3789fff635e441258b7b9d3f6d not found: ID does not exist"
May 28 00:57:55.859368 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.859322    2550 scope.go:117] "RemoveContainer" containerID="038427d942c06b56c779c54973a238cdfe41b4acfe97c72c9bf0f02673f45ec1"
May 28 00:57:55.859615 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.859596    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"038427d942c06b56c779c54973a238cdfe41b4acfe97c72c9bf0f02673f45ec1"} err="failed to get container status \"038427d942c06b56c779c54973a238cdfe41b4acfe97c72c9bf0f02673f45ec1\": rpc error: code = NotFound desc = could not find container \"038427d942c06b56c779c54973a238cdfe41b4acfe97c72c9bf0f02673f45ec1\": container with ID starting with 038427d942c06b56c779c54973a238cdfe41b4acfe97c72c9bf0f02673f45ec1 not found: ID does not exist"
May 28 00:57:55.859674 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.859615    2550 scope.go:117] "RemoveContainer" containerID="93a79c73015262201ae9eb4fac3e6c70ec11303357518a986e07127dadb70744"
May 28 00:57:55.859851 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.859824    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93a79c73015262201ae9eb4fac3e6c70ec11303357518a986e07127dadb70744"} err="failed to get container status \"93a79c73015262201ae9eb4fac3e6c70ec11303357518a986e07127dadb70744\": rpc error: code = NotFound desc = could not find container \"93a79c73015262201ae9eb4fac3e6c70ec11303357518a986e07127dadb70744\": container with ID starting with 93a79c73015262201ae9eb4fac3e6c70ec11303357518a986e07127dadb70744 not found: ID does not exist"
May 28 00:57:55.859851 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.859846    2550 scope.go:117] "RemoveContainer" containerID="a2d93e2bd2928c65dbc9321aa428ad1c00e64727931ef48363a45234f8625abf"
May 28 00:57:55.860061 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.860045    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2d93e2bd2928c65dbc9321aa428ad1c00e64727931ef48363a45234f8625abf"} err="failed to get container status \"a2d93e2bd2928c65dbc9321aa428ad1c00e64727931ef48363a45234f8625abf\": rpc error: code = NotFound desc = could not find container \"a2d93e2bd2928c65dbc9321aa428ad1c00e64727931ef48363a45234f8625abf\": container with ID starting with a2d93e2bd2928c65dbc9321aa428ad1c00e64727931ef48363a45234f8625abf not found: ID does not exist"
May 28 00:57:55.860061 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.860061    2550 scope.go:117] "RemoveContainer" containerID="981d77e03484b01e7da205fea04ce8ed7ec62503b70687d28875ab003e1c047c"
May 28 00:57:55.860251 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.860236    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"981d77e03484b01e7da205fea04ce8ed7ec62503b70687d28875ab003e1c047c"} err="failed to get container status \"981d77e03484b01e7da205fea04ce8ed7ec62503b70687d28875ab003e1c047c\": rpc error: code = NotFound desc = could not find container \"981d77e03484b01e7da205fea04ce8ed7ec62503b70687d28875ab003e1c047c\": container with ID starting with 981d77e03484b01e7da205fea04ce8ed7ec62503b70687d28875ab003e1c047c not found: ID does not exist"
May 28 00:57:55.862217 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.862193    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 00:57:55.862439 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.862409    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerName="alertmanager"
May 28 00:57:55.862514 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.862441    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerName="alertmanager"
May 28 00:57:55.862514 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.862458    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerName="init-config-reloader"
May 28 00:57:55.862514 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.862464    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerName="init-config-reloader"
May 28 00:57:55.862514 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.862470    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerName="config-reloader"
May 28 00:57:55.862514 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.862475    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerName="config-reloader"
May 28 00:57:55.862514 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.862482    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerName="kube-rbac-proxy-web"
May 28 00:57:55.862514 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.862486    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerName="kube-rbac-proxy-web"
May 28 00:57:55.862514 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.862492    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerName="kube-rbac-proxy-metric"
May 28 00:57:55.862514 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.862497    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerName="kube-rbac-proxy-metric"
May 28 00:57:55.862514 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.862506    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerName="kube-rbac-proxy"
May 28 00:57:55.862514 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.862511    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerName="kube-rbac-proxy"
May 28 00:57:55.862514 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.862517    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerName="prom-label-proxy"
May 28 00:57:55.862514 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.862522    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerName="prom-label-proxy"
May 28 00:57:55.863106 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.862559    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerName="kube-rbac-proxy-metric"
May 28 00:57:55.863106 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.862567    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerName="config-reloader"
May 28 00:57:55.863106 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.862575    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerName="kube-rbac-proxy-web"
May 28 00:57:55.863106 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.862581    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerName="kube-rbac-proxy"
May 28 00:57:55.863106 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.862589    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerName="alertmanager"
May 28 00:57:55.863106 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.862595    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" containerName="prom-label-proxy"
May 28 00:57:55.867295 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.867278    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:55.869502 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.869480    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
May 28 00:57:55.869595 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.869526    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
May 28 00:57:55.869595 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.869531    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
May 28 00:57:55.869682 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.869549    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
May 28 00:57:55.869728 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.869662    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
May 28 00:57:55.869979 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.869963    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
May 28 00:57:55.870068 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.869997    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
May 28 00:57:55.870230 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.870214    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-dknrw\""
May 28 00:57:55.870452 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.870406    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
May 28 00:57:55.874669 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.874649    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
May 28 00:57:55.878613 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.878588    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 00:57:55.961961 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.961918    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0c7ad1ef-47c3-4758-8177-e329a2a73a61-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:55.961961 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.961969    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0c7ad1ef-47c3-4758-8177-e329a2a73a61-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:55.962181 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.961993    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0c7ad1ef-47c3-4758-8177-e329a2a73a61-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:55.962181 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.962012    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0c7ad1ef-47c3-4758-8177-e329a2a73a61-web-config\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:55.962181 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.962043    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlx7p\" (UniqueName: \"kubernetes.io/projected/0c7ad1ef-47c3-4758-8177-e329a2a73a61-kube-api-access-wlx7p\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:55.962181 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.962087    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0c7ad1ef-47c3-4758-8177-e329a2a73a61-config-volume\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:55.962181 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.962109    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0c7ad1ef-47c3-4758-8177-e329a2a73a61-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:55.962181 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.962129    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0c7ad1ef-47c3-4758-8177-e329a2a73a61-config-out\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:55.962181 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.962154    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c7ad1ef-47c3-4758-8177-e329a2a73a61-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:55.962181 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.962181    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0c7ad1ef-47c3-4758-8177-e329a2a73a61-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:55.962450 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.962198    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0c7ad1ef-47c3-4758-8177-e329a2a73a61-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:55.962450 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.962224    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0c7ad1ef-47c3-4758-8177-e329a2a73a61-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:55.962450 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:55.962245    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c7ad1ef-47c3-4758-8177-e329a2a73a61-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.063282 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.063228    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0c7ad1ef-47c3-4758-8177-e329a2a73a61-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.063282 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.063283    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0c7ad1ef-47c3-4758-8177-e329a2a73a61-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.063492 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.063307    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0c7ad1ef-47c3-4758-8177-e329a2a73a61-web-config\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.063557 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.063537    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wlx7p\" (UniqueName: \"kubernetes.io/projected/0c7ad1ef-47c3-4758-8177-e329a2a73a61-kube-api-access-wlx7p\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.063601 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.063574    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0c7ad1ef-47c3-4758-8177-e329a2a73a61-config-volume\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.063601 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.063593    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0c7ad1ef-47c3-4758-8177-e329a2a73a61-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.063701 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.063613    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0c7ad1ef-47c3-4758-8177-e329a2a73a61-config-out\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.063701 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.063633    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c7ad1ef-47c3-4758-8177-e329a2a73a61-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.063701 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.063680    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0c7ad1ef-47c3-4758-8177-e329a2a73a61-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.063835 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.063711    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0c7ad1ef-47c3-4758-8177-e329a2a73a61-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.063835 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.063739    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0c7ad1ef-47c3-4758-8177-e329a2a73a61-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.063835 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.063763    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c7ad1ef-47c3-4758-8177-e329a2a73a61-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.063835 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.063819    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0c7ad1ef-47c3-4758-8177-e329a2a73a61-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.064016 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.063968    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0c7ad1ef-47c3-4758-8177-e329a2a73a61-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.065151 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.065117    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c7ad1ef-47c3-4758-8177-e329a2a73a61-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.065288 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.065152    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c7ad1ef-47c3-4758-8177-e329a2a73a61-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.066510 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.066442    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0c7ad1ef-47c3-4758-8177-e329a2a73a61-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.066602 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.066526    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0c7ad1ef-47c3-4758-8177-e329a2a73a61-web-config\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.066602 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.066537    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0c7ad1ef-47c3-4758-8177-e329a2a73a61-config-volume\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.066893 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.066870    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0c7ad1ef-47c3-4758-8177-e329a2a73a61-config-out\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.067056 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.067029    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0c7ad1ef-47c3-4758-8177-e329a2a73a61-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.067143 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.067127    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0c7ad1ef-47c3-4758-8177-e329a2a73a61-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.067208 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.067181    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0c7ad1ef-47c3-4758-8177-e329a2a73a61-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.067398 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.067376    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0c7ad1ef-47c3-4758-8177-e329a2a73a61-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.068265 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.068248    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0c7ad1ef-47c3-4758-8177-e329a2a73a61-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.072320 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.072298    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlx7p\" (UniqueName: \"kubernetes.io/projected/0c7ad1ef-47c3-4758-8177-e329a2a73a61-kube-api-access-wlx7p\") pod \"alertmanager-main-0\" (UID: \"0c7ad1ef-47c3-4758-8177-e329a2a73a61\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.176624 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.176591    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 28 00:57:56.311685 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.311649    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 00:57:56.315922 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:57:56.315889    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c7ad1ef_47c3_4758_8177_e329a2a73a61.slice/crio-463cde15d6844eab3b45df41e4f6530fb11907f77b4537651ea4300eb4d0ef52 WatchSource:0}: Error finding container 463cde15d6844eab3b45df41e4f6530fb11907f77b4537651ea4300eb4d0ef52: Status 404 returned error can't find the container with id 463cde15d6844eab3b45df41e4f6530fb11907f77b4537651ea4300eb4d0ef52
May 28 00:57:56.817171 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.817133    2550 generic.go:358] "Generic (PLEG): container finished" podID="0c7ad1ef-47c3-4758-8177-e329a2a73a61" containerID="0f56dbb75deb73b0f4be044c2c359abc043102f0a51e6b6952e93a3fadf05b48" exitCode=0
May 28 00:57:56.817581 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.817218    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c7ad1ef-47c3-4758-8177-e329a2a73a61","Type":"ContainerDied","Data":"0f56dbb75deb73b0f4be044c2c359abc043102f0a51e6b6952e93a3fadf05b48"}
May 28 00:57:56.817581 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:56.817258    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c7ad1ef-47c3-4758-8177-e329a2a73a61","Type":"ContainerStarted","Data":"463cde15d6844eab3b45df41e4f6530fb11907f77b4537651ea4300eb4d0ef52"}
May 28 00:57:57.324984 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:57.324885    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d0f24db-4a2d-4454-a9c2-5d87a855f953" path="/var/lib/kubelet/pods/2d0f24db-4a2d-4454-a9c2-5d87a855f953/volumes"
May 28 00:57:57.824215 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:57.824179    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c7ad1ef-47c3-4758-8177-e329a2a73a61","Type":"ContainerStarted","Data":"8e918e654380f4c4d3df69db5562a28bfb835dc1aa310fdd1586550927911f43"}
May 28 00:57:57.824215 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:57.824218    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c7ad1ef-47c3-4758-8177-e329a2a73a61","Type":"ContainerStarted","Data":"bd3e938e040b5480d1553a5934f822272c03e69629c5bc807677afcb629d154e"}
May 28 00:57:57.824719 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:57.824236    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c7ad1ef-47c3-4758-8177-e329a2a73a61","Type":"ContainerStarted","Data":"7b2bc965b5448a66aa4a926f2fb9241d565ff6f4e8bd90279d1979ec3901d771"}
May 28 00:57:57.824719 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:57.824248    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c7ad1ef-47c3-4758-8177-e329a2a73a61","Type":"ContainerStarted","Data":"e80cfcd905b4ce109e1f2dc53530a7ff2eed30f1be0ae9c09a5bdbb14397a760"}
May 28 00:57:57.824719 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:57.824260    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c7ad1ef-47c3-4758-8177-e329a2a73a61","Type":"ContainerStarted","Data":"a2c99cc93548ac043964fc0605151a9900e142642b084b14c3e83ae6e11f6ab4"}
May 28 00:57:57.824719 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:57.824272    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0c7ad1ef-47c3-4758-8177-e329a2a73a61","Type":"ContainerStarted","Data":"7c6b64fdf79361530fcf802417a5f54a703be46b8756503ba622f6e480d362b1"}
May 28 00:57:57.852034 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:57.851982    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.851968105 podStartE2EDuration="2.851968105s" podCreationTimestamp="2026-05-28 00:57:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 00:57:57.850629529 +0000 UTC m=+137.131586637" watchObservedRunningTime="2026-05-28 00:57:57.851968105 +0000 UTC m=+137.132925197"
May 28 00:57:58.294114 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.294075    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-54554f4f95-prf65"]
May 28 00:57:58.297936 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.297914    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-54554f4f95-prf65"
May 28 00:57:58.300560 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.300519    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
May 28 00:57:58.300560 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.300519    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
May 28 00:57:58.300751 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.300686    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
May 28 00:57:58.300751 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.300716    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-w6xl2\""
May 28 00:57:58.300751 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.300731    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
May 28 00:57:58.301024 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.301006    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
May 28 00:57:58.306011 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.305991    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
May 28 00:57:58.308409 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.308387    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-54554f4f95-prf65"]
May 28 00:57:58.383689 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.383647    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f778803e-1d0e-4983-84ab-33f29e7b141a-secret-telemeter-client\") pod \"telemeter-client-54554f4f95-prf65\" (UID: \"f778803e-1d0e-4983-84ab-33f29e7b141a\") " pod="openshift-monitoring/telemeter-client-54554f4f95-prf65"
May 28 00:57:58.383845 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.383700    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmsvs\" (UniqueName: \"kubernetes.io/projected/f778803e-1d0e-4983-84ab-33f29e7b141a-kube-api-access-rmsvs\") pod \"telemeter-client-54554f4f95-prf65\" (UID: \"f778803e-1d0e-4983-84ab-33f29e7b141a\") " pod="openshift-monitoring/telemeter-client-54554f4f95-prf65"
May 28 00:57:58.383845 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.383723    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f778803e-1d0e-4983-84ab-33f29e7b141a-federate-client-tls\") pod \"telemeter-client-54554f4f95-prf65\" (UID: \"f778803e-1d0e-4983-84ab-33f29e7b141a\") " pod="openshift-monitoring/telemeter-client-54554f4f95-prf65"
May 28 00:57:58.383845 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.383807    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f778803e-1d0e-4983-84ab-33f29e7b141a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-54554f4f95-prf65\" (UID: \"f778803e-1d0e-4983-84ab-33f29e7b141a\") " pod="openshift-monitoring/telemeter-client-54554f4f95-prf65"
May 28 00:57:58.383958 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.383851    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f778803e-1d0e-4983-84ab-33f29e7b141a-serving-certs-ca-bundle\") pod \"telemeter-client-54554f4f95-prf65\" (UID: \"f778803e-1d0e-4983-84ab-33f29e7b141a\") " pod="openshift-monitoring/telemeter-client-54554f4f95-prf65"
May 28 00:57:58.383958 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.383884    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f778803e-1d0e-4983-84ab-33f29e7b141a-telemeter-client-tls\") pod \"telemeter-client-54554f4f95-prf65\" (UID: \"f778803e-1d0e-4983-84ab-33f29e7b141a\") " pod="openshift-monitoring/telemeter-client-54554f4f95-prf65"
May 28 00:57:58.383958 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.383908    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f778803e-1d0e-4983-84ab-33f29e7b141a-metrics-client-ca\") pod \"telemeter-client-54554f4f95-prf65\" (UID: \"f778803e-1d0e-4983-84ab-33f29e7b141a\") " pod="openshift-monitoring/telemeter-client-54554f4f95-prf65"
May 28 00:57:58.383958 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.383931    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f778803e-1d0e-4983-84ab-33f29e7b141a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-54554f4f95-prf65\" (UID: \"f778803e-1d0e-4983-84ab-33f29e7b141a\") " pod="openshift-monitoring/telemeter-client-54554f4f95-prf65"
May 28 00:57:58.484855 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.484798    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rmsvs\" (UniqueName: \"kubernetes.io/projected/f778803e-1d0e-4983-84ab-33f29e7b141a-kube-api-access-rmsvs\") pod \"telemeter-client-54554f4f95-prf65\" (UID: \"f778803e-1d0e-4983-84ab-33f29e7b141a\") " pod="openshift-monitoring/telemeter-client-54554f4f95-prf65"
May 28 00:57:58.484855 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.484866    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f778803e-1d0e-4983-84ab-33f29e7b141a-federate-client-tls\") pod \"telemeter-client-54554f4f95-prf65\" (UID: \"f778803e-1d0e-4983-84ab-33f29e7b141a\") " pod="openshift-monitoring/telemeter-client-54554f4f95-prf65"
May 28 00:57:58.485115 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.484903    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f778803e-1d0e-4983-84ab-33f29e7b141a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-54554f4f95-prf65\" (UID: \"f778803e-1d0e-4983-84ab-33f29e7b141a\") " pod="openshift-monitoring/telemeter-client-54554f4f95-prf65"
May 28 00:57:58.485115 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.484928    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f778803e-1d0e-4983-84ab-33f29e7b141a-serving-certs-ca-bundle\") pod \"telemeter-client-54554f4f95-prf65\" (UID: \"f778803e-1d0e-4983-84ab-33f29e7b141a\") " pod="openshift-monitoring/telemeter-client-54554f4f95-prf65"
May 28 00:57:58.485115 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.484951    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f778803e-1d0e-4983-84ab-33f29e7b141a-telemeter-client-tls\") pod \"telemeter-client-54554f4f95-prf65\" (UID: \"f778803e-1d0e-4983-84ab-33f29e7b141a\") " pod="openshift-monitoring/telemeter-client-54554f4f95-prf65"
May 28 00:57:58.485115 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.484973    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f778803e-1d0e-4983-84ab-33f29e7b141a-metrics-client-ca\") pod \"telemeter-client-54554f4f95-prf65\" (UID: \"f778803e-1d0e-4983-84ab-33f29e7b141a\") " pod="openshift-monitoring/telemeter-client-54554f4f95-prf65"
May 28 00:57:58.485115 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.484997    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f778803e-1d0e-4983-84ab-33f29e7b141a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-54554f4f95-prf65\" (UID: \"f778803e-1d0e-4983-84ab-33f29e7b141a\") " pod="openshift-monitoring/telemeter-client-54554f4f95-prf65"
May 28 00:57:58.485115 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.485039    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f778803e-1d0e-4983-84ab-33f29e7b141a-secret-telemeter-client\") pod \"telemeter-client-54554f4f95-prf65\" (UID: \"f778803e-1d0e-4983-84ab-33f29e7b141a\") " pod="openshift-monitoring/telemeter-client-54554f4f95-prf65"
May 28 00:57:58.485881 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.485850    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f778803e-1d0e-4983-84ab-33f29e7b141a-serving-certs-ca-bundle\") pod \"telemeter-client-54554f4f95-prf65\" (UID: \"f778803e-1d0e-4983-84ab-33f29e7b141a\") " pod="openshift-monitoring/telemeter-client-54554f4f95-prf65"
May 28 00:57:58.486049 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.486023    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f778803e-1d0e-4983-84ab-33f29e7b141a-metrics-client-ca\") pod \"telemeter-client-54554f4f95-prf65\" (UID: \"f778803e-1d0e-4983-84ab-33f29e7b141a\") " pod="openshift-monitoring/telemeter-client-54554f4f95-prf65"
May 28 00:57:58.486136 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.486033    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f778803e-1d0e-4983-84ab-33f29e7b141a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-54554f4f95-prf65\" (UID: \"f778803e-1d0e-4983-84ab-33f29e7b141a\") " pod="openshift-monitoring/telemeter-client-54554f4f95-prf65"
May 28 00:57:58.487614 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.487592    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f778803e-1d0e-4983-84ab-33f29e7b141a-secret-telemeter-client\") pod \"telemeter-client-54554f4f95-prf65\" (UID: \"f778803e-1d0e-4983-84ab-33f29e7b141a\") " pod="openshift-monitoring/telemeter-client-54554f4f95-prf65"
May 28 00:57:58.487769 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.487748    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f778803e-1d0e-4983-84ab-33f29e7b141a-federate-client-tls\") pod \"telemeter-client-54554f4f95-prf65\" (UID: \"f778803e-1d0e-4983-84ab-33f29e7b141a\") " pod="openshift-monitoring/telemeter-client-54554f4f95-prf65"
May 28 00:57:58.487882 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.487864    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f778803e-1d0e-4983-84ab-33f29e7b141a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-54554f4f95-prf65\" (UID: \"f778803e-1d0e-4983-84ab-33f29e7b141a\") " pod="openshift-monitoring/telemeter-client-54554f4f95-prf65"
May 28 00:57:58.487959 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.487939    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f778803e-1d0e-4983-84ab-33f29e7b141a-telemeter-client-tls\") pod \"telemeter-client-54554f4f95-prf65\" (UID: \"f778803e-1d0e-4983-84ab-33f29e7b141a\") " pod="openshift-monitoring/telemeter-client-54554f4f95-prf65"
May 28 00:57:58.492337 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.492296    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmsvs\" (UniqueName: \"kubernetes.io/projected/f778803e-1d0e-4983-84ab-33f29e7b141a-kube-api-access-rmsvs\") pod \"telemeter-client-54554f4f95-prf65\" (UID: \"f778803e-1d0e-4983-84ab-33f29e7b141a\") " pod="openshift-monitoring/telemeter-client-54554f4f95-prf65"
May 28 00:57:58.608808 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.608725    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-54554f4f95-prf65"
May 28 00:57:58.741672 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.741636    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-54554f4f95-prf65"]
May 28 00:57:58.744542 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:57:58.744501    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf778803e_1d0e_4983_84ab_33f29e7b141a.slice/crio-69b105de44c2f22e734252b8255ae1f4247a2a9ea87bd858cee6c312e410379a WatchSource:0}: Error finding container 69b105de44c2f22e734252b8255ae1f4247a2a9ea87bd858cee6c312e410379a: Status 404 returned error can't find the container with id 69b105de44c2f22e734252b8255ae1f4247a2a9ea87bd858cee6c312e410379a
May 28 00:57:58.828245 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:57:58.828210    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54554f4f95-prf65" event={"ID":"f778803e-1d0e-4983-84ab-33f29e7b141a","Type":"ContainerStarted","Data":"69b105de44c2f22e734252b8255ae1f4247a2a9ea87bd858cee6c312e410379a"}
May 28 00:58:00.836095 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:00.836061    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54554f4f95-prf65" event={"ID":"f778803e-1d0e-4983-84ab-33f29e7b141a","Type":"ContainerStarted","Data":"2f5eeab038ec22fc2fb5f10ffc2fece2def9ec7ce2469420ba602683adeab672"}
May 28 00:58:00.836095 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:00.836101    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54554f4f95-prf65" event={"ID":"f778803e-1d0e-4983-84ab-33f29e7b141a","Type":"ContainerStarted","Data":"4283d90d9b5c34bda49296b8d3b44bd95b45b1775553dd3f77de4dec519263f6"}
May 28 00:58:00.836639 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:00.836117    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54554f4f95-prf65" event={"ID":"f778803e-1d0e-4983-84ab-33f29e7b141a","Type":"ContainerStarted","Data":"31cbe0e06fb14ca56f1bee8f836067d0e2349a7a34de78ad6e49f96eb1d71219"}
May 28 00:58:00.860339 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:00.860283    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-54554f4f95-prf65" podStartSLOduration=0.96125765 podStartE2EDuration="2.860266147s" podCreationTimestamp="2026-05-28 00:57:58 +0000 UTC" firstStartedPulling="2026-05-28 00:57:58.746467763 +0000 UTC m=+138.027424833" lastFinishedPulling="2026-05-28 00:58:00.645476259 +0000 UTC m=+139.926433330" observedRunningTime="2026-05-28 00:58:00.857983072 +0000 UTC m=+140.138940165" watchObservedRunningTime="2026-05-28 00:58:00.860266147 +0000 UTC m=+140.141223242"
May 28 00:58:01.718568 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:01.718530    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-f9d6fbb94-pmmwk"]
May 28 00:58:01.721785 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:01.721761    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:01.731491 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:01.731465    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f9d6fbb94-pmmwk"]
May 28 00:58:01.812094 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:01.812051    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-console-config\") pod \"console-f9d6fbb94-pmmwk\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") " pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:01.812094 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:01.812098    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-oauth-serving-cert\") pod \"console-f9d6fbb94-pmmwk\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") " pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:01.812298 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:01.812116    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt4qg\" (UniqueName: \"kubernetes.io/projected/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-kube-api-access-wt4qg\") pod \"console-f9d6fbb94-pmmwk\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") " pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:01.812298 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:01.812135    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-service-ca\") pod \"console-f9d6fbb94-pmmwk\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") " pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:01.812298 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:01.812204    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-trusted-ca-bundle\") pod \"console-f9d6fbb94-pmmwk\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") " pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:01.812298 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:01.812274    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-console-oauth-config\") pod \"console-f9d6fbb94-pmmwk\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") " pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:01.812473 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:01.812313    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-console-serving-cert\") pod \"console-f9d6fbb94-pmmwk\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") " pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:01.913066 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:01.913020    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-console-oauth-config\") pod \"console-f9d6fbb94-pmmwk\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") " pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:01.913704 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:01.913112    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-console-serving-cert\") pod \"console-f9d6fbb94-pmmwk\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") " pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:01.913704 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:01.913143    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-console-config\") pod \"console-f9d6fbb94-pmmwk\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") " pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:01.913704 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:01.913193    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-oauth-serving-cert\") pod \"console-f9d6fbb94-pmmwk\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") " pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:01.913704 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:01.913217    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wt4qg\" (UniqueName: \"kubernetes.io/projected/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-kube-api-access-wt4qg\") pod \"console-f9d6fbb94-pmmwk\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") " pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:01.913704 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:01.913250    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-service-ca\") pod \"console-f9d6fbb94-pmmwk\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") " pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:01.913704 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:01.913302    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-trusted-ca-bundle\") pod \"console-f9d6fbb94-pmmwk\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") " pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:01.914235 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:01.914205    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-oauth-serving-cert\") pod \"console-f9d6fbb94-pmmwk\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") " pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:01.914326 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:01.914246    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-service-ca\") pod \"console-f9d6fbb94-pmmwk\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") " pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:01.914362 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:01.914326    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-trusted-ca-bundle\") pod \"console-f9d6fbb94-pmmwk\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") " pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:01.914392 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:01.914373    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-console-config\") pod \"console-f9d6fbb94-pmmwk\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") " pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:01.916129 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:01.916108    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-console-serving-cert\") pod \"console-f9d6fbb94-pmmwk\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") " pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:01.916247 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:01.916227    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-console-oauth-config\") pod \"console-f9d6fbb94-pmmwk\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") " pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:01.922107 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:01.922073    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt4qg\" (UniqueName: \"kubernetes.io/projected/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-kube-api-access-wt4qg\") pod \"console-f9d6fbb94-pmmwk\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") " pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:02.031835 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:02.031741    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:02.159907 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:02.159875    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f9d6fbb94-pmmwk"]
May 28 00:58:02.162687 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:58:02.162650    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf904e8f1_3dd4_496b_a4c4_cbbb988794ce.slice/crio-e231e0589edbe690551b882ea7362edccfee0d3c68d039fdcd979ecf6adad41b WatchSource:0}: Error finding container e231e0589edbe690551b882ea7362edccfee0d3c68d039fdcd979ecf6adad41b: Status 404 returned error can't find the container with id e231e0589edbe690551b882ea7362edccfee0d3c68d039fdcd979ecf6adad41b
May 28 00:58:02.843650 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:02.843613    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f9d6fbb94-pmmwk" event={"ID":"f904e8f1-3dd4-496b-a4c4-cbbb988794ce","Type":"ContainerStarted","Data":"f504e9f972ae1d4a6839cb303a1a65dd45f801646d0983af29628d95946eeeee"}
May 28 00:58:02.843650 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:02.843651    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f9d6fbb94-pmmwk" event={"ID":"f904e8f1-3dd4-496b-a4c4-cbbb988794ce","Type":"ContainerStarted","Data":"e231e0589edbe690551b882ea7362edccfee0d3c68d039fdcd979ecf6adad41b"}
May 28 00:58:02.861634 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:02.861585    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-f9d6fbb94-pmmwk" podStartSLOduration=1.861569062 podStartE2EDuration="1.861569062s" podCreationTimestamp="2026-05-28 00:58:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 00:58:02.861096026 +0000 UTC m=+142.142053119" watchObservedRunningTime="2026-05-28 00:58:02.861569062 +0000 UTC m=+142.142526163"
May 28 00:58:12.032788 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:12.032739    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:12.033291 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:12.032808    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:12.037651 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:12.037628    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:12.875430 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:12.875393    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:58:12.934283 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:12.934252    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f6bdcb5cd-mhp4d"]
May 28 00:58:37.954936 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:37.954874    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-7f6bdcb5cd-mhp4d" podUID="e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3" containerName="console" containerID="cri-o://449689a6008b562aeceb9f68d17215377cf91cbec41f097898b6ae836848f086" gracePeriod=15
May 28 00:58:38.198924 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.198901    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7f6bdcb5cd-mhp4d_e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3/console/0.log"
May 28 00:58:38.199055 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.198963    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:58:38.210308 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.210246    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-console-serving-cert\") pod \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") "
May 28 00:58:38.210308 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.210285    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-service-ca\") pod \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") "
May 28 00:58:38.210464 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.210321    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2r4w\" (UniqueName: \"kubernetes.io/projected/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-kube-api-access-p2r4w\") pod \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") "
May 28 00:58:38.210464 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.210336    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-console-oauth-config\") pod \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") "
May 28 00:58:38.210572 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.210468    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-oauth-serving-cert\") pod \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") "
May 28 00:58:38.210572 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.210525    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-trusted-ca-bundle\") pod \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") "
May 28 00:58:38.210572 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.210549    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-console-config\") pod \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\" (UID: \"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3\") "
May 28 00:58:38.211180 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.210756    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-service-ca" (OuterVolumeSpecName: "service-ca") pod "e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3" (UID: "e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 00:58:38.211180 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.210863    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3" (UID: "e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 00:58:38.211180 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.210896    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3" (UID: "e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 00:58:38.211180 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.211128    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-console-config" (OuterVolumeSpecName: "console-config") pod "e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3" (UID: "e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 00:58:38.212783 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.212754    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3" (UID: "e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 00:58:38.212872 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.212801    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-kube-api-access-p2r4w" (OuterVolumeSpecName: "kube-api-access-p2r4w") pod "e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3" (UID: "e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3"). InnerVolumeSpecName "kube-api-access-p2r4w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 00:58:38.212872 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.212801    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3" (UID: "e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 00:58:38.311852 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.311811    2550 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-service-ca\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:58:38.311852 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.311850    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2r4w\" (UniqueName: \"kubernetes.io/projected/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-kube-api-access-p2r4w\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:58:38.311852 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.311864    2550 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-console-oauth-config\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:58:38.312085 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.311874    2550 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-oauth-serving-cert\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:58:38.312085 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.311883    2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-trusted-ca-bundle\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:58:38.312085 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.311892    2550 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-console-config\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:58:38.312085 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.311901    2550 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3-console-serving-cert\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:58:38.944202 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.944174    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7f6bdcb5cd-mhp4d_e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3/console/0.log"
May 28 00:58:38.944373 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.944216    2550 generic.go:358] "Generic (PLEG): container finished" podID="e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3" containerID="449689a6008b562aeceb9f68d17215377cf91cbec41f097898b6ae836848f086" exitCode=2
May 28 00:58:38.944373 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.944256    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f6bdcb5cd-mhp4d" event={"ID":"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3","Type":"ContainerDied","Data":"449689a6008b562aeceb9f68d17215377cf91cbec41f097898b6ae836848f086"}
May 28 00:58:38.944373 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.944289    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f6bdcb5cd-mhp4d"
May 28 00:58:38.944373 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.944308    2550 scope.go:117] "RemoveContainer" containerID="449689a6008b562aeceb9f68d17215377cf91cbec41f097898b6ae836848f086"
May 28 00:58:38.944373 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.944295    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f6bdcb5cd-mhp4d" event={"ID":"e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3","Type":"ContainerDied","Data":"8f2fcb6e7c7b7dc995db4cb98a98544b7bb727c9e892088ed5e827abfb4bbbb2"}
May 28 00:58:38.952301 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.952276    2550 scope.go:117] "RemoveContainer" containerID="449689a6008b562aeceb9f68d17215377cf91cbec41f097898b6ae836848f086"
May 28 00:58:38.952611 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:58:38.952589    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"449689a6008b562aeceb9f68d17215377cf91cbec41f097898b6ae836848f086\": container with ID starting with 449689a6008b562aeceb9f68d17215377cf91cbec41f097898b6ae836848f086 not found: ID does not exist" containerID="449689a6008b562aeceb9f68d17215377cf91cbec41f097898b6ae836848f086"
May 28 00:58:38.952672 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.952620    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"449689a6008b562aeceb9f68d17215377cf91cbec41f097898b6ae836848f086"} err="failed to get container status \"449689a6008b562aeceb9f68d17215377cf91cbec41f097898b6ae836848f086\": rpc error: code = NotFound desc = could not find container \"449689a6008b562aeceb9f68d17215377cf91cbec41f097898b6ae836848f086\": container with ID starting with 449689a6008b562aeceb9f68d17215377cf91cbec41f097898b6ae836848f086 not found: ID does not exist"
May 28 00:58:38.964628 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.964598    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f6bdcb5cd-mhp4d"]
May 28 00:58:38.966617 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:38.966594    2550 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f6bdcb5cd-mhp4d"]
May 28 00:58:39.323758 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:58:39.323682    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3" path="/var/lib/kubelet/pods/e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3/volumes"
May 28 00:59:18.150396 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.150343    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-7796dc9f79-zxtp4"]
May 28 00:59:18.150864 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.150648    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3" containerName="console"
May 28 00:59:18.150864 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.150660    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3" containerName="console"
May 28 00:59:18.150864 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.150703    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="e3d47ac1-5bf7-4da1-831f-01b1b1ef55e3" containerName="console"
May 28 00:59:18.153820 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.153797    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:18.164822 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.164797    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7796dc9f79-zxtp4"]
May 28 00:59:18.226186 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.226153    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-oauth-serving-cert\") pod \"console-7796dc9f79-zxtp4\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") " pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:18.226361 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.226217    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-console-config\") pod \"console-7796dc9f79-zxtp4\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") " pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:18.226361 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.226246    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-service-ca\") pod \"console-7796dc9f79-zxtp4\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") " pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:18.226361 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.226265    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-console-oauth-config\") pod \"console-7796dc9f79-zxtp4\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") " pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:18.226361 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.226286    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4c55\" (UniqueName: \"kubernetes.io/projected/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-kube-api-access-g4c55\") pod \"console-7796dc9f79-zxtp4\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") " pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:18.226532 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.226371    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-trusted-ca-bundle\") pod \"console-7796dc9f79-zxtp4\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") " pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:18.226532 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.226442    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-console-serving-cert\") pod \"console-7796dc9f79-zxtp4\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") " pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:18.327786 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.327745    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-console-serving-cert\") pod \"console-7796dc9f79-zxtp4\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") " pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:18.327901 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.327808    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-oauth-serving-cert\") pod \"console-7796dc9f79-zxtp4\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") " pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:18.327901 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.327864    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-console-config\") pod \"console-7796dc9f79-zxtp4\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") " pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:18.327999 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.327900    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-service-ca\") pod \"console-7796dc9f79-zxtp4\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") " pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:18.327999 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.327927    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-console-oauth-config\") pod \"console-7796dc9f79-zxtp4\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") " pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:18.327999 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.327957    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g4c55\" (UniqueName: \"kubernetes.io/projected/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-kube-api-access-g4c55\") pod \"console-7796dc9f79-zxtp4\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") " pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:18.327999 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.327988    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-trusted-ca-bundle\") pod \"console-7796dc9f79-zxtp4\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") " pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:18.328685 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.328658    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-oauth-serving-cert\") pod \"console-7796dc9f79-zxtp4\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") " pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:18.328823 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.328772    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-console-config\") pod \"console-7796dc9f79-zxtp4\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") " pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:18.328823 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.328774    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-service-ca\") pod \"console-7796dc9f79-zxtp4\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") " pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:18.328941 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.328867    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-trusted-ca-bundle\") pod \"console-7796dc9f79-zxtp4\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") " pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:18.330488 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.330464    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-console-oauth-config\") pod \"console-7796dc9f79-zxtp4\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") " pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:18.330581 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.330466    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-console-serving-cert\") pod \"console-7796dc9f79-zxtp4\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") " pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:18.336640 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.336619    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4c55\" (UniqueName: \"kubernetes.io/projected/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-kube-api-access-g4c55\") pod \"console-7796dc9f79-zxtp4\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") " pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:18.463263 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.463233    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:18.592366 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:18.592329    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7796dc9f79-zxtp4"]
May 28 00:59:18.595523 ip-10-0-131-73 kubenswrapper[2550]: W0528 00:59:18.595493    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8ea8cca_dfac_48a3_8cce_73aa5f3db3dc.slice/crio-cd257f59be62c3d8a3772d631c5402ea1cf18293dcaba2f527c1a7527e10e025 WatchSource:0}: Error finding container cd257f59be62c3d8a3772d631c5402ea1cf18293dcaba2f527c1a7527e10e025: Status 404 returned error can't find the container with id cd257f59be62c3d8a3772d631c5402ea1cf18293dcaba2f527c1a7527e10e025
May 28 00:59:19.054941 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:19.054893    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7796dc9f79-zxtp4" event={"ID":"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc","Type":"ContainerStarted","Data":"2f014f9aa4f39e24d4217286ff66acf83c9cf06921c1b549b73827b0d933b1c1"}
May 28 00:59:19.054941 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:19.054945    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7796dc9f79-zxtp4" event={"ID":"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc","Type":"ContainerStarted","Data":"cd257f59be62c3d8a3772d631c5402ea1cf18293dcaba2f527c1a7527e10e025"}
May 28 00:59:19.074846 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:19.074785    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-7796dc9f79-zxtp4" podStartSLOduration=1.074753955 podStartE2EDuration="1.074753955s" podCreationTimestamp="2026-05-28 00:59:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 00:59:19.073607532 +0000 UTC m=+218.354564626" watchObservedRunningTime="2026-05-28 00:59:19.074753955 +0000 UTC m=+218.355711049"
May 28 00:59:28.464141 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:28.464107    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:28.464604 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:28.464160    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:28.468813 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:28.468786    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:29.086616 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:29.086588    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 00:59:29.134359 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:29.134324    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f9d6fbb94-pmmwk"]
May 28 00:59:54.153082 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:54.152999    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-f9d6fbb94-pmmwk" podUID="f904e8f1-3dd4-496b-a4c4-cbbb988794ce" containerName="console" containerID="cri-o://f504e9f972ae1d4a6839cb303a1a65dd45f801646d0983af29628d95946eeeee" gracePeriod=15
May 28 00:59:54.393934 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:54.393910    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-f9d6fbb94-pmmwk_f904e8f1-3dd4-496b-a4c4-cbbb988794ce/console/0.log"
May 28 00:59:54.394045 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:54.393975    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:59:54.505255 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:54.505220    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-console-serving-cert\") pod \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") "
May 28 00:59:54.505457 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:54.505273    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-console-config\") pod \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") "
May 28 00:59:54.505457 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:54.505301    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-trusted-ca-bundle\") pod \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") "
May 28 00:59:54.505560 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:54.505499    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-console-oauth-config\") pod \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") "
May 28 00:59:54.505615 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:54.505556    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-service-ca\") pod \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") "
May 28 00:59:54.505615 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:54.505594    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-oauth-serving-cert\") pod \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") "
May 28 00:59:54.505717 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:54.505624    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wt4qg\" (UniqueName: \"kubernetes.io/projected/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-kube-api-access-wt4qg\") pod \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\" (UID: \"f904e8f1-3dd4-496b-a4c4-cbbb988794ce\") "
May 28 00:59:54.505771 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:54.505705    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-console-config" (OuterVolumeSpecName: "console-config") pod "f904e8f1-3dd4-496b-a4c4-cbbb988794ce" (UID: "f904e8f1-3dd4-496b-a4c4-cbbb988794ce"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 00:59:54.505771 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:54.505726    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f904e8f1-3dd4-496b-a4c4-cbbb988794ce" (UID: "f904e8f1-3dd4-496b-a4c4-cbbb988794ce"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 00:59:54.505972 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:54.505937    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-service-ca" (OuterVolumeSpecName: "service-ca") pod "f904e8f1-3dd4-496b-a4c4-cbbb988794ce" (UID: "f904e8f1-3dd4-496b-a4c4-cbbb988794ce"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 00:59:54.505972 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:54.505955    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f904e8f1-3dd4-496b-a4c4-cbbb988794ce" (UID: "f904e8f1-3dd4-496b-a4c4-cbbb988794ce"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 00:59:54.506047 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:54.505991    2550 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-console-config\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:59:54.506047 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:54.506011    2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-trusted-ca-bundle\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:59:54.507452 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:54.507427    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f904e8f1-3dd4-496b-a4c4-cbbb988794ce" (UID: "f904e8f1-3dd4-496b-a4c4-cbbb988794ce"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 00:59:54.507589 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:54.507573    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-kube-api-access-wt4qg" (OuterVolumeSpecName: "kube-api-access-wt4qg") pod "f904e8f1-3dd4-496b-a4c4-cbbb988794ce" (UID: "f904e8f1-3dd4-496b-a4c4-cbbb988794ce"). InnerVolumeSpecName "kube-api-access-wt4qg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 00:59:54.507628 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:54.507581    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f904e8f1-3dd4-496b-a4c4-cbbb988794ce" (UID: "f904e8f1-3dd4-496b-a4c4-cbbb988794ce"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 00:59:54.607086 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:54.607055    2550 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-console-serving-cert\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:59:54.607086 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:54.607081    2550 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-console-oauth-config\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:59:54.607086 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:54.607092    2550 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-service-ca\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:59:54.607305 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:54.607101    2550 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-oauth-serving-cert\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:59:54.607305 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:54.607110    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wt4qg\" (UniqueName: \"kubernetes.io/projected/f904e8f1-3dd4-496b-a4c4-cbbb988794ce-kube-api-access-wt4qg\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 00:59:55.157544 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:55.157517    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-f9d6fbb94-pmmwk_f904e8f1-3dd4-496b-a4c4-cbbb988794ce/console/0.log"
May 28 00:59:55.157918 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:55.157558    2550 generic.go:358] "Generic (PLEG): container finished" podID="f904e8f1-3dd4-496b-a4c4-cbbb988794ce" containerID="f504e9f972ae1d4a6839cb303a1a65dd45f801646d0983af29628d95946eeeee" exitCode=2
May 28 00:59:55.157918 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:55.157595    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f9d6fbb94-pmmwk" event={"ID":"f904e8f1-3dd4-496b-a4c4-cbbb988794ce","Type":"ContainerDied","Data":"f504e9f972ae1d4a6839cb303a1a65dd45f801646d0983af29628d95946eeeee"}
May 28 00:59:55.157918 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:55.157628    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f9d6fbb94-pmmwk"
May 28 00:59:55.157918 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:55.157640    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f9d6fbb94-pmmwk" event={"ID":"f904e8f1-3dd4-496b-a4c4-cbbb988794ce","Type":"ContainerDied","Data":"e231e0589edbe690551b882ea7362edccfee0d3c68d039fdcd979ecf6adad41b"}
May 28 00:59:55.157918 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:55.157657    2550 scope.go:117] "RemoveContainer" containerID="f504e9f972ae1d4a6839cb303a1a65dd45f801646d0983af29628d95946eeeee"
May 28 00:59:55.165655 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:55.165637    2550 scope.go:117] "RemoveContainer" containerID="f504e9f972ae1d4a6839cb303a1a65dd45f801646d0983af29628d95946eeeee"
May 28 00:59:55.165924 ip-10-0-131-73 kubenswrapper[2550]: E0528 00:59:55.165908    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f504e9f972ae1d4a6839cb303a1a65dd45f801646d0983af29628d95946eeeee\": container with ID starting with f504e9f972ae1d4a6839cb303a1a65dd45f801646d0983af29628d95946eeeee not found: ID does not exist" containerID="f504e9f972ae1d4a6839cb303a1a65dd45f801646d0983af29628d95946eeeee"
May 28 00:59:55.165974 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:55.165931    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f504e9f972ae1d4a6839cb303a1a65dd45f801646d0983af29628d95946eeeee"} err="failed to get container status \"f504e9f972ae1d4a6839cb303a1a65dd45f801646d0983af29628d95946eeeee\": rpc error: code = NotFound desc = could not find container \"f504e9f972ae1d4a6839cb303a1a65dd45f801646d0983af29628d95946eeeee\": container with ID starting with f504e9f972ae1d4a6839cb303a1a65dd45f801646d0983af29628d95946eeeee not found: ID does not exist"
May 28 00:59:55.178502 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:55.178479    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f9d6fbb94-pmmwk"]
May 28 00:59:55.182145 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:55.182110    2550 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f9d6fbb94-pmmwk"]
May 28 00:59:55.323757 ip-10-0-131-73 kubenswrapper[2550]: I0528 00:59:55.323717    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f904e8f1-3dd4-496b-a4c4-cbbb988794ce" path="/var/lib/kubelet/pods/f904e8f1-3dd4-496b-a4c4-cbbb988794ce/volumes"
May 28 01:00:41.226981 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:00:41.226945    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9z98m_6671c1c1-b674-4a1d-9011-1ca22724b234/ovn-acl-logging/0.log"
May 28 01:00:41.228348 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:00:41.228323    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9z98m_6671c1c1-b674-4a1d-9011-1ca22724b234/ovn-acl-logging/0.log"
May 28 01:00:41.231228 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:00:41.231207    2550 kubelet.go:1628] "Image garbage collection succeeded"
May 28 01:01:13.519644 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:13.519610    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5d796b8684-jqqqx"]
May 28 01:01:13.522023 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:13.519869    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f904e8f1-3dd4-496b-a4c4-cbbb988794ce" containerName="console"
May 28 01:01:13.522023 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:13.519879    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="f904e8f1-3dd4-496b-a4c4-cbbb988794ce" containerName="console"
May 28 01:01:13.522023 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:13.519931    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="f904e8f1-3dd4-496b-a4c4-cbbb988794ce" containerName="console"
May 28 01:01:13.522910 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:13.522894    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5d796b8684-jqqqx"
May 28 01:01:13.525021 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:13.524999    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\""
May 28 01:01:13.525120 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:13.525002    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\""
May 28 01:01:13.525503 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:13.525487    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-8mrkf\""
May 28 01:01:13.527630 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:13.527610    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5d796b8684-jqqqx"]
May 28 01:01:13.631900 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:13.631863    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8w8w\" (UniqueName: \"kubernetes.io/projected/0fe53fb5-edc8-47fe-9a2c-b1d076959f64-kube-api-access-t8w8w\") pod \"feast-operator-controller-manager-5d796b8684-jqqqx\" (UID: \"0fe53fb5-edc8-47fe-9a2c-b1d076959f64\") " pod="feast-operator-system/feast-operator-controller-manager-5d796b8684-jqqqx"
May 28 01:01:13.732708 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:13.732669    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t8w8w\" (UniqueName: \"kubernetes.io/projected/0fe53fb5-edc8-47fe-9a2c-b1d076959f64-kube-api-access-t8w8w\") pod \"feast-operator-controller-manager-5d796b8684-jqqqx\" (UID: \"0fe53fb5-edc8-47fe-9a2c-b1d076959f64\") " pod="feast-operator-system/feast-operator-controller-manager-5d796b8684-jqqqx"
May 28 01:01:13.741345 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:13.741322    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8w8w\" (UniqueName: \"kubernetes.io/projected/0fe53fb5-edc8-47fe-9a2c-b1d076959f64-kube-api-access-t8w8w\") pod \"feast-operator-controller-manager-5d796b8684-jqqqx\" (UID: \"0fe53fb5-edc8-47fe-9a2c-b1d076959f64\") " pod="feast-operator-system/feast-operator-controller-manager-5d796b8684-jqqqx"
May 28 01:01:13.833458 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:13.833350    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5d796b8684-jqqqx"
May 28 01:01:13.954925 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:13.954892    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5d796b8684-jqqqx"]
May 28 01:01:13.957963 ip-10-0-131-73 kubenswrapper[2550]: W0528 01:01:13.957937    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fe53fb5_edc8_47fe_9a2c_b1d076959f64.slice/crio-7f734598383b09812c2ed7edca3990dd4dbb6bf4fd39ef1f53767c4d8b2f0586 WatchSource:0}: Error finding container 7f734598383b09812c2ed7edca3990dd4dbb6bf4fd39ef1f53767c4d8b2f0586: Status 404 returned error can't find the container with id 7f734598383b09812c2ed7edca3990dd4dbb6bf4fd39ef1f53767c4d8b2f0586
May 28 01:01:13.959701 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:13.959686    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 01:01:14.378221 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:14.378185    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5d796b8684-jqqqx" event={"ID":"0fe53fb5-edc8-47fe-9a2c-b1d076959f64","Type":"ContainerStarted","Data":"7f734598383b09812c2ed7edca3990dd4dbb6bf4fd39ef1f53767c4d8b2f0586"}
May 28 01:01:17.387516 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:17.387480    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5d796b8684-jqqqx" event={"ID":"0fe53fb5-edc8-47fe-9a2c-b1d076959f64","Type":"ContainerStarted","Data":"be234907ccdfa351764a14c3925fc89820afe3e7406783d24214d4e302aaa053"}
May 28 01:01:17.387852 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:17.387606    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-5d796b8684-jqqqx"
May 28 01:01:17.403221 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:17.403166    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-5d796b8684-jqqqx" podStartSLOduration=1.053861009 podStartE2EDuration="4.403146817s" podCreationTimestamp="2026-05-28 01:01:13 +0000 UTC" firstStartedPulling="2026-05-28 01:01:13.959809994 +0000 UTC m=+333.240767066" lastFinishedPulling="2026-05-28 01:01:17.3090958 +0000 UTC m=+336.590052874" observedRunningTime="2026-05-28 01:01:17.40273801 +0000 UTC m=+336.683695103" watchObservedRunningTime="2026-05-28 01:01:17.403146817 +0000 UTC m=+336.684103912"
May 28 01:01:28.392764 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:28.392696    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-5d796b8684-jqqqx"
May 28 01:01:56.418842 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.418800    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"]
May 28 01:01:56.422265 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.422248    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:01:56.424547 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.424522    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 28 01:01:56.424903 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.424883    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 28 01:01:56.425000 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.424958    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-77tpz\""
May 28 01:01:56.425000 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.424893    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 28 01:01:56.425136 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.424996    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 28 01:01:56.425136 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.425019    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 28 01:01:56.425136 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.424884    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 28 01:01:56.425285 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.425237    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\""
May 28 01:01:56.432142 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.432120    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"]
May 28 01:01:56.454590 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.454562    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-online-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-mhwzx\" (UID: \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:01:56.454730 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.454603    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-ui-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-mhwzx\" (UID: \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:01:56.454730 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.454690    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhz6n\" (UniqueName: \"kubernetes.io/projected/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-kube-api-access-vhz6n\") pod \"feast-simple-feast-setup-6ff8bdb7f4-mhwzx\" (UID: \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:01:56.454841 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.454737    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-feast-data\") pod \"feast-simple-feast-setup-6ff8bdb7f4-mhwzx\" (UID: \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:01:56.454841 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.454775    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-offline-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-mhwzx\" (UID: \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:01:56.454841 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.454818    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-registry-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-mhwzx\" (UID: \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:01:56.555403 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.555370    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vhz6n\" (UniqueName: \"kubernetes.io/projected/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-kube-api-access-vhz6n\") pod \"feast-simple-feast-setup-6ff8bdb7f4-mhwzx\" (UID: \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:01:56.555403 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.555406    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-feast-data\") pod \"feast-simple-feast-setup-6ff8bdb7f4-mhwzx\" (UID: \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:01:56.555665 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.555448    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-offline-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-mhwzx\" (UID: \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:01:56.555665 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.555495    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-registry-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-mhwzx\" (UID: \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:01:56.555665 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.555522    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-online-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-mhwzx\" (UID: \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:01:56.555665 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:01:56.555616    2550 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found
May 28 01:01:56.555665 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.555660    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-ui-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-mhwzx\" (UID: \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:01:56.555893 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:01:56.555688    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-registry-tls podName:dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0 nodeName:}" failed. No retries permitted until 2026-05-28 01:01:57.055667414 +0000 UTC m=+376.336624484 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-registry-tls") pod "feast-simple-feast-setup-6ff8bdb7f4-mhwzx" (UID: "dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0") : secret "feast-simple-feast-setup-registry-tls" not found
May 28 01:01:56.555893 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:01:56.555729    2550 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 28 01:01:56.555893 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:01:56.555763    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-ui-tls podName:dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0 nodeName:}" failed. No retries permitted until 2026-05-28 01:01:57.055752001 +0000 UTC m=+376.336709075 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-ui-tls") pod "feast-simple-feast-setup-6ff8bdb7f4-mhwzx" (UID: "dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0") : secret "feast-simple-feast-setup-ui-tls" not found
May 28 01:01:56.555893 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.555810    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-feast-data\") pod \"feast-simple-feast-setup-6ff8bdb7f4-mhwzx\" (UID: \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:01:56.557837 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.557815    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-online-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-mhwzx\" (UID: \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:01:56.557952 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.557843    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-offline-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-mhwzx\" (UID: \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:01:56.563746 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:56.563725    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhz6n\" (UniqueName: \"kubernetes.io/projected/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-kube-api-access-vhz6n\") pod \"feast-simple-feast-setup-6ff8bdb7f4-mhwzx\" (UID: \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:01:57.060535 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:57.060502    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-registry-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-mhwzx\" (UID: \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:01:57.060726 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:57.060548    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-ui-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-mhwzx\" (UID: \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:01:57.062953 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:57.062931    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-registry-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-mhwzx\" (UID: \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:01:57.063067 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:57.062937    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-ui-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-mhwzx\" (UID: \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:01:57.340078 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:57.340006    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:01:57.474940 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:57.474912    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"]
May 28 01:01:57.476377 ip-10-0-131-73 kubenswrapper[2550]: W0528 01:01:57.476351    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddcd634b8_b6bf_46af_ad7c_d6c5bab5efd0.slice/crio-3f6b40d5997ef3f160214c83cf259e1e2f620f3604e53cab64b570039b1b319f WatchSource:0}: Error finding container 3f6b40d5997ef3f160214c83cf259e1e2f620f3604e53cab64b570039b1b319f: Status 404 returned error can't find the container with id 3f6b40d5997ef3f160214c83cf259e1e2f620f3604e53cab64b570039b1b319f
May 28 01:01:57.494916 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:01:57.494886    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" event={"ID":"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0","Type":"ContainerStarted","Data":"3f6b40d5997ef3f160214c83cf259e1e2f620f3604e53cab64b570039b1b319f"}
May 28 01:02:32.593716 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:32.593677    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" event={"ID":"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0","Type":"ContainerStarted","Data":"103f468642c988fbcf2c26e343b18e3507ce26c4cbe40dc1d8659cdab981912f"}
May 28 01:02:36.606975 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:36.606882    2550 generic.go:358] "Generic (PLEG): container finished" podID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerID="103f468642c988fbcf2c26e343b18e3507ce26c4cbe40dc1d8659cdab981912f" exitCode=0
May 28 01:02:36.606975 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:36.606929    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" event={"ID":"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0","Type":"ContainerDied","Data":"103f468642c988fbcf2c26e343b18e3507ce26c4cbe40dc1d8659cdab981912f"}
May 28 01:02:37.611141 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:37.611103    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" event={"ID":"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0","Type":"ContainerStarted","Data":"285f85cc5698b5217a12332c00df4cc13af7fe17f2f0ccb1be90eaac0f49986f"}
May 28 01:02:42.626962 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:42.626927    2550 generic.go:358] "Generic (PLEG): container finished" podID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerID="285f85cc5698b5217a12332c00df4cc13af7fe17f2f0ccb1be90eaac0f49986f" exitCode=0
May 28 01:02:42.627332 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:42.626995    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" event={"ID":"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0","Type":"ContainerDied","Data":"285f85cc5698b5217a12332c00df4cc13af7fe17f2f0ccb1be90eaac0f49986f"}
May 28 01:02:42.635673 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:42.635652    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\""
May 28 01:02:43.634561 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:43.634521    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" event={"ID":"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0","Type":"ContainerStarted","Data":"cc345928d00e2109b6b1efbdfbedfbc59804b96023c6d4a9c1b6b6d7eda06c08"}
May 28 01:02:43.635040 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:43.634576    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" event={"ID":"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0","Type":"ContainerStarted","Data":"cbb7ff6b5d2e237f3ced00d165fa1c9020c26b0275582cdfbdc898ecd097f733"}
May 28 01:02:43.635040 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:43.634593    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" event={"ID":"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0","Type":"ContainerStarted","Data":"89111c246559d82e931d6a697bdbf223d73572012c409603d6edc40e5f6d1684"}
May 28 01:02:43.635040 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:43.634607    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" event={"ID":"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0","Type":"ContainerStarted","Data":"5d108bfc0052e6f7332bc2040f6b7686a99252fed456657e8337325bfb0795fa"}
May 28 01:02:43.679340 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:43.679273    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" podStartSLOduration=13.438906928 podStartE2EDuration="47.679250861s" podCreationTimestamp="2026-05-28 01:01:56 +0000 UTC" firstStartedPulling="2026-05-28 01:01:57.478190987 +0000 UTC m=+376.759148057" lastFinishedPulling="2026-05-28 01:02:31.718534906 +0000 UTC m=+410.999491990" observedRunningTime="2026-05-28 01:02:43.676776635 +0000 UTC m=+422.957733730" watchObservedRunningTime="2026-05-28 01:02:43.679250861 +0000 UTC m=+422.960207955"
May 28 01:02:45.340542 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:45.340501    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:02:45.341109 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:45.341077    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:02:45.341109 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:45.341106    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:02:45.341299 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:45.341120    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:02:45.342901 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:45.342860    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.18:6571: connect: connection refused"
May 28 01:02:45.343124 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:45.342884    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.18:8443: connect: connection refused"
May 28 01:02:45.343225 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:45.342919    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.18:8816: connect: connection refused"
May 28 01:02:45.343225 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:45.342909    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.18:6567/health\": dial tcp 10.133.0.18:6567: connect: connection refused"
May 28 01:02:47.340424 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:47.340361    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:02:47.340898 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:47.340437    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:02:47.340898 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:47.340456    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:02:47.340898 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:47.340468    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:02:48.341487 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:48.341360    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:02:48.341943 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:48.341593    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:02:48.341943 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:48.341800    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:02:48.342175 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:48.342156    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:02:48.342240 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:48.342202    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:02:48.346085 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:48.346061    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:02:48.655329 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:48.655241    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:02:48.657794 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:02:48.657774    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:03:11.085618 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.085522    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d855856b4-7qkd5"]
May 28 01:03:11.173054 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.173019    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d855856b4-7qkd5"]
May 28 01:03:11.173216 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.173169    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:11.287129 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.287085    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4a9b823-a3a8-43b5-b620-a436a598a939-console-config\") pod \"console-5d855856b4-7qkd5\" (UID: \"d4a9b823-a3a8-43b5-b620-a436a598a939\") " pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:11.287129 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.287136    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4a9b823-a3a8-43b5-b620-a436a598a939-service-ca\") pod \"console-5d855856b4-7qkd5\" (UID: \"d4a9b823-a3a8-43b5-b620-a436a598a939\") " pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:11.287355 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.287152    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4a9b823-a3a8-43b5-b620-a436a598a939-trusted-ca-bundle\") pod \"console-5d855856b4-7qkd5\" (UID: \"d4a9b823-a3a8-43b5-b620-a436a598a939\") " pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:11.287355 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.287172    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7ksb\" (UniqueName: \"kubernetes.io/projected/d4a9b823-a3a8-43b5-b620-a436a598a939-kube-api-access-s7ksb\") pod \"console-5d855856b4-7qkd5\" (UID: \"d4a9b823-a3a8-43b5-b620-a436a598a939\") " pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:11.287355 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.287188    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4a9b823-a3a8-43b5-b620-a436a598a939-console-oauth-config\") pod \"console-5d855856b4-7qkd5\" (UID: \"d4a9b823-a3a8-43b5-b620-a436a598a939\") " pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:11.287355 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.287206    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4a9b823-a3a8-43b5-b620-a436a598a939-oauth-serving-cert\") pod \"console-5d855856b4-7qkd5\" (UID: \"d4a9b823-a3a8-43b5-b620-a436a598a939\") " pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:11.287355 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.287274    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4a9b823-a3a8-43b5-b620-a436a598a939-console-serving-cert\") pod \"console-5d855856b4-7qkd5\" (UID: \"d4a9b823-a3a8-43b5-b620-a436a598a939\") " pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:11.388273 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.388189    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4a9b823-a3a8-43b5-b620-a436a598a939-console-serving-cert\") pod \"console-5d855856b4-7qkd5\" (UID: \"d4a9b823-a3a8-43b5-b620-a436a598a939\") " pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:11.388273 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.388242    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4a9b823-a3a8-43b5-b620-a436a598a939-console-config\") pod \"console-5d855856b4-7qkd5\" (UID: \"d4a9b823-a3a8-43b5-b620-a436a598a939\") " pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:11.388273 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.388271    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4a9b823-a3a8-43b5-b620-a436a598a939-service-ca\") pod \"console-5d855856b4-7qkd5\" (UID: \"d4a9b823-a3a8-43b5-b620-a436a598a939\") " pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:11.388610 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.388285    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4a9b823-a3a8-43b5-b620-a436a598a939-trusted-ca-bundle\") pod \"console-5d855856b4-7qkd5\" (UID: \"d4a9b823-a3a8-43b5-b620-a436a598a939\") " pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:11.388610 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.388303    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s7ksb\" (UniqueName: \"kubernetes.io/projected/d4a9b823-a3a8-43b5-b620-a436a598a939-kube-api-access-s7ksb\") pod \"console-5d855856b4-7qkd5\" (UID: \"d4a9b823-a3a8-43b5-b620-a436a598a939\") " pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:11.388610 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.388478    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4a9b823-a3a8-43b5-b620-a436a598a939-console-oauth-config\") pod \"console-5d855856b4-7qkd5\" (UID: \"d4a9b823-a3a8-43b5-b620-a436a598a939\") " pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:11.388610 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.388519    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4a9b823-a3a8-43b5-b620-a436a598a939-oauth-serving-cert\") pod \"console-5d855856b4-7qkd5\" (UID: \"d4a9b823-a3a8-43b5-b620-a436a598a939\") " pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:11.389103 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.389074    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4a9b823-a3a8-43b5-b620-a436a598a939-console-config\") pod \"console-5d855856b4-7qkd5\" (UID: \"d4a9b823-a3a8-43b5-b620-a436a598a939\") " pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:11.389289 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.389183    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4a9b823-a3a8-43b5-b620-a436a598a939-service-ca\") pod \"console-5d855856b4-7qkd5\" (UID: \"d4a9b823-a3a8-43b5-b620-a436a598a939\") " pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:11.389289 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.389183    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4a9b823-a3a8-43b5-b620-a436a598a939-oauth-serving-cert\") pod \"console-5d855856b4-7qkd5\" (UID: \"d4a9b823-a3a8-43b5-b620-a436a598a939\") " pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:11.389462 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.389385    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4a9b823-a3a8-43b5-b620-a436a598a939-trusted-ca-bundle\") pod \"console-5d855856b4-7qkd5\" (UID: \"d4a9b823-a3a8-43b5-b620-a436a598a939\") " pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:11.390838 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.390818    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4a9b823-a3a8-43b5-b620-a436a598a939-console-serving-cert\") pod \"console-5d855856b4-7qkd5\" (UID: \"d4a9b823-a3a8-43b5-b620-a436a598a939\") " pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:11.391076 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.391055    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4a9b823-a3a8-43b5-b620-a436a598a939-console-oauth-config\") pod \"console-5d855856b4-7qkd5\" (UID: \"d4a9b823-a3a8-43b5-b620-a436a598a939\") " pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:11.396097 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.396074    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7ksb\" (UniqueName: \"kubernetes.io/projected/d4a9b823-a3a8-43b5-b620-a436a598a939-kube-api-access-s7ksb\") pod \"console-5d855856b4-7qkd5\" (UID: \"d4a9b823-a3a8-43b5-b620-a436a598a939\") " pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:11.482142 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.482114    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:11.614622 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.614588    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d855856b4-7qkd5"]
May 28 01:03:11.617616 ip-10-0-131-73 kubenswrapper[2550]: W0528 01:03:11.617589    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4a9b823_a3a8_43b5_b620_a436a598a939.slice/crio-7778134027cec581068faabc186a603d3ffd91b1f7cbe36bbe961c7586d3125f WatchSource:0}: Error finding container 7778134027cec581068faabc186a603d3ffd91b1f7cbe36bbe961c7586d3125f: Status 404 returned error can't find the container with id 7778134027cec581068faabc186a603d3ffd91b1f7cbe36bbe961c7586d3125f
May 28 01:03:11.720626 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.720590    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d855856b4-7qkd5" event={"ID":"d4a9b823-a3a8-43b5-b620-a436a598a939","Type":"ContainerStarted","Data":"4b1e2bef9da64a1546865e82a2ae45fd12a037012012ff89f86b1739465feec1"}
May 28 01:03:11.720793 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.720633    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d855856b4-7qkd5" event={"ID":"d4a9b823-a3a8-43b5-b620-a436a598a939","Type":"ContainerStarted","Data":"7778134027cec581068faabc186a603d3ffd91b1f7cbe36bbe961c7586d3125f"}
May 28 01:03:11.737456 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:11.737382    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-5d855856b4-7qkd5" podStartSLOduration=0.737367238 podStartE2EDuration="737.367238ms" podCreationTimestamp="2026-05-28 01:03:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 01:03:11.736174765 +0000 UTC m=+451.017131880" watchObservedRunningTime="2026-05-28 01:03:11.737367238 +0000 UTC m=+451.018324331"
May 28 01:03:21.482902 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:21.482851    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:21.482902 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:21.482902    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:21.487587 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:21.487563    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:21.755453 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:21.755365    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d855856b4-7qkd5"
May 28 01:03:21.799199 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:21.799171    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7796dc9f79-zxtp4"]
May 28 01:03:46.817750 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:46.817685    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-7796dc9f79-zxtp4" podUID="c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc" containerName="console" containerID="cri-o://2f014f9aa4f39e24d4217286ff66acf83c9cf06921c1b549b73827b0d933b1c1" gracePeriod=15
May 28 01:03:47.053038 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.053012    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7796dc9f79-zxtp4_c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc/console/0.log"
May 28 01:03:47.053162 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.053077    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 01:03:47.091455 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.091362    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-console-config\") pod \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") "
May 28 01:03:47.091455 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.091426    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-service-ca\") pod \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") "
May 28 01:03:47.091660 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.091459    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-console-serving-cert\") pod \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") "
May 28 01:03:47.091660 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.091498    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-trusted-ca-bundle\") pod \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") "
May 28 01:03:47.091660 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.091530    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4c55\" (UniqueName: \"kubernetes.io/projected/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-kube-api-access-g4c55\") pod \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") "
May 28 01:03:47.091660 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.091567    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-oauth-serving-cert\") pod \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") "
May 28 01:03:47.091660 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.091615    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-console-oauth-config\") pod \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\" (UID: \"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc\") "
May 28 01:03:47.091895 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.091844    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-console-config" (OuterVolumeSpecName: "console-config") pod "c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc" (UID: "c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 01:03:47.091960 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.091919    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-service-ca" (OuterVolumeSpecName: "service-ca") pod "c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc" (UID: "c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 01:03:47.092153 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.092126    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc" (UID: "c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 01:03:47.092276 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.092184    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc" (UID: "c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 01:03:47.094096 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.094064    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc" (UID: "c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:03:47.094096 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.094076    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc" (UID: "c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:03:47.094221 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.094190    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-kube-api-access-g4c55" (OuterVolumeSpecName: "kube-api-access-g4c55") pod "c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc" (UID: "c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc"). InnerVolumeSpecName "kube-api-access-g4c55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 01:03:47.192964 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.192927    2550 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-service-ca\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:03:47.192964 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.192958    2550 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-console-serving-cert\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:03:47.192964 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.192969    2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-trusted-ca-bundle\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:03:47.193191 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.192978    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4c55\" (UniqueName: \"kubernetes.io/projected/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-kube-api-access-g4c55\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:03:47.193191 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.192987    2550 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-oauth-serving-cert\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:03:47.193191 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.192994    2550 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-console-oauth-config\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:03:47.193191 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.193004    2550 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc-console-config\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:03:47.826664 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.826638    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7796dc9f79-zxtp4_c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc/console/0.log"
May 28 01:03:47.827073 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.826675    2550 generic.go:358] "Generic (PLEG): container finished" podID="c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc" containerID="2f014f9aa4f39e24d4217286ff66acf83c9cf06921c1b549b73827b0d933b1c1" exitCode=2
May 28 01:03:47.827073 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.826742    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7796dc9f79-zxtp4"
May 28 01:03:47.827073 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.826749    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7796dc9f79-zxtp4" event={"ID":"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc","Type":"ContainerDied","Data":"2f014f9aa4f39e24d4217286ff66acf83c9cf06921c1b549b73827b0d933b1c1"}
May 28 01:03:47.827073 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.826775    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7796dc9f79-zxtp4" event={"ID":"c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc","Type":"ContainerDied","Data":"cd257f59be62c3d8a3772d631c5402ea1cf18293dcaba2f527c1a7527e10e025"}
May 28 01:03:47.827073 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.826790    2550 scope.go:117] "RemoveContainer" containerID="2f014f9aa4f39e24d4217286ff66acf83c9cf06921c1b549b73827b0d933b1c1"
May 28 01:03:47.834991 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.834958    2550 scope.go:117] "RemoveContainer" containerID="2f014f9aa4f39e24d4217286ff66acf83c9cf06921c1b549b73827b0d933b1c1"
May 28 01:03:47.835262 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:03:47.835242    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f014f9aa4f39e24d4217286ff66acf83c9cf06921c1b549b73827b0d933b1c1\": container with ID starting with 2f014f9aa4f39e24d4217286ff66acf83c9cf06921c1b549b73827b0d933b1c1 not found: ID does not exist" containerID="2f014f9aa4f39e24d4217286ff66acf83c9cf06921c1b549b73827b0d933b1c1"
May 28 01:03:47.835318 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.835280    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f014f9aa4f39e24d4217286ff66acf83c9cf06921c1b549b73827b0d933b1c1"} err="failed to get container status \"2f014f9aa4f39e24d4217286ff66acf83c9cf06921c1b549b73827b0d933b1c1\": rpc error: code = NotFound desc = could not find container \"2f014f9aa4f39e24d4217286ff66acf83c9cf06921c1b549b73827b0d933b1c1\": container with ID starting with 2f014f9aa4f39e24d4217286ff66acf83c9cf06921c1b549b73827b0d933b1c1 not found: ID does not exist"
May 28 01:03:47.858958 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.858924    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7796dc9f79-zxtp4"]
May 28 01:03:47.863388 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:47.863360    2550 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7796dc9f79-zxtp4"]
May 28 01:03:49.323255 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:49.323222    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc" path="/var/lib/kubelet/pods/c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc/volumes"
May 28 01:03:51.673394 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:51.673361    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"]
May 28 01:03:51.674360 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:51.674298    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="registry" containerID="cri-o://5d108bfc0052e6f7332bc2040f6b7686a99252fed456657e8337325bfb0795fa" gracePeriod=30
May 28 01:03:51.674771 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:51.674745    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="ui" containerID="cri-o://cc345928d00e2109b6b1efbdfbedfbc59804b96023c6d4a9c1b6b6d7eda06c08" gracePeriod=30
May 28 01:03:51.674867 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:51.674850    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="offline" containerID="cri-o://cbb7ff6b5d2e237f3ced00d165fa1c9020c26b0275582cdfbdc898ecd097f733" gracePeriod=30
May 28 01:03:51.674935 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:51.674917    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="online" containerID="cri-o://89111c246559d82e931d6a697bdbf223d73572012c409603d6edc40e5f6d1684" gracePeriod=30
May 28 01:03:51.950122 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:51.950040    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"]
May 28 01:03:51.950496 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:51.950475    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc" containerName="console"
May 28 01:03:51.950537 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:51.950499    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc" containerName="console"
May 28 01:03:51.950606 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:51.950595    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="c8ea8cca-dfac-48a3-8cce-73aa5f3db3dc" containerName="console"
May 28 01:03:51.955978 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:51.955954    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:03:51.958043 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:51.958023    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-gm56j\""
May 28 01:03:51.964526 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:51.964503    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"]
May 28 01:03:52.031802 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:52.031769    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-offline-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-755df\" (UID: \"e427384d-69e1-4cfc-b796-9f35349b668c\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:03:52.031802 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:52.031810    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-registry-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-755df\" (UID: \"e427384d-69e1-4cfc-b796-9f35349b668c\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:03:52.032040 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:52.031842    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e427384d-69e1-4cfc-b796-9f35349b668c-feast-data\") pod \"feast-simple-feast-setup-6ff8bdb7f4-755df\" (UID: \"e427384d-69e1-4cfc-b796-9f35349b668c\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:03:52.032040 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:52.031865    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj575\" (UniqueName: \"kubernetes.io/projected/e427384d-69e1-4cfc-b796-9f35349b668c-kube-api-access-vj575\") pod \"feast-simple-feast-setup-6ff8bdb7f4-755df\" (UID: \"e427384d-69e1-4cfc-b796-9f35349b668c\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:03:52.032040 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:52.031907    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-ui-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-755df\" (UID: \"e427384d-69e1-4cfc-b796-9f35349b668c\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:03:52.032040 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:52.031931    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-online-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-755df\" (UID: \"e427384d-69e1-4cfc-b796-9f35349b668c\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:03:52.132929 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:52.132888    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e427384d-69e1-4cfc-b796-9f35349b668c-feast-data\") pod \"feast-simple-feast-setup-6ff8bdb7f4-755df\" (UID: \"e427384d-69e1-4cfc-b796-9f35349b668c\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:03:52.132929 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:52.132932    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vj575\" (UniqueName: \"kubernetes.io/projected/e427384d-69e1-4cfc-b796-9f35349b668c-kube-api-access-vj575\") pod \"feast-simple-feast-setup-6ff8bdb7f4-755df\" (UID: \"e427384d-69e1-4cfc-b796-9f35349b668c\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:03:52.133171 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:52.132975    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-ui-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-755df\" (UID: \"e427384d-69e1-4cfc-b796-9f35349b668c\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:03:52.133171 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:52.133027    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-online-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-755df\" (UID: \"e427384d-69e1-4cfc-b796-9f35349b668c\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:03:52.133171 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:52.133116    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-offline-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-755df\" (UID: \"e427384d-69e1-4cfc-b796-9f35349b668c\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:03:52.133171 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:52.133145    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-registry-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-755df\" (UID: \"e427384d-69e1-4cfc-b796-9f35349b668c\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:03:52.133388 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:03:52.133249    2550 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found
May 28 01:03:52.133388 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:03:52.133341    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-offline-tls podName:e427384d-69e1-4cfc-b796-9f35349b668c nodeName:}" failed. No retries permitted until 2026-05-28 01:03:52.633316421 +0000 UTC m=+491.914273511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-offline-tls") pod "feast-simple-feast-setup-6ff8bdb7f4-755df" (UID: "e427384d-69e1-4cfc-b796-9f35349b668c") : secret "feast-simple-feast-setup-offline-tls" not found
May 28 01:03:52.133647 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:52.133614    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e427384d-69e1-4cfc-b796-9f35349b668c-feast-data\") pod \"feast-simple-feast-setup-6ff8bdb7f4-755df\" (UID: \"e427384d-69e1-4cfc-b796-9f35349b668c\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:03:52.135638 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:52.135609    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-ui-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-755df\" (UID: \"e427384d-69e1-4cfc-b796-9f35349b668c\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:03:52.135765 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:52.135675    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-registry-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-755df\" (UID: \"e427384d-69e1-4cfc-b796-9f35349b668c\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:03:52.135904 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:52.135884    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-online-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-755df\" (UID: \"e427384d-69e1-4cfc-b796-9f35349b668c\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:03:52.142216 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:52.142193    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj575\" (UniqueName: \"kubernetes.io/projected/e427384d-69e1-4cfc-b796-9f35349b668c-kube-api-access-vj575\") pod \"feast-simple-feast-setup-6ff8bdb7f4-755df\" (UID: \"e427384d-69e1-4cfc-b796-9f35349b668c\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:03:52.637066 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:52.636974    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-offline-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-755df\" (UID: \"e427384d-69e1-4cfc-b796-9f35349b668c\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:03:52.639275 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:52.639257    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-offline-tls\") pod \"feast-simple-feast-setup-6ff8bdb7f4-755df\" (UID: \"e427384d-69e1-4cfc-b796-9f35349b668c\") " pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:03:52.844467 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:52.844432    2550 generic.go:358] "Generic (PLEG): container finished" podID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerID="cc345928d00e2109b6b1efbdfbedfbc59804b96023c6d4a9c1b6b6d7eda06c08" exitCode=0
May 28 01:03:52.844467 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:52.844460    2550 generic.go:358] "Generic (PLEG): container finished" podID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerID="89111c246559d82e931d6a697bdbf223d73572012c409603d6edc40e5f6d1684" exitCode=0
May 28 01:03:52.844864 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:52.844486    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" event={"ID":"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0","Type":"ContainerDied","Data":"cc345928d00e2109b6b1efbdfbedfbc59804b96023c6d4a9c1b6b6d7eda06c08"}
May 28 01:03:52.844864 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:52.844520    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" event={"ID":"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0","Type":"ContainerDied","Data":"89111c246559d82e931d6a697bdbf223d73572012c409603d6edc40e5f6d1684"}
May 28 01:03:52.866719 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:52.866686    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:03:52.999585 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:52.999555    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"]
May 28 01:03:53.001636 ip-10-0-131-73 kubenswrapper[2550]: W0528 01:03:53.001609    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode427384d_69e1_4cfc_b796_9f35349b668c.slice/crio-c8784fd857a21999d37199e98ce5b37c4fcf6f780d41c15100b9a90b0c9f173e WatchSource:0}: Error finding container c8784fd857a21999d37199e98ce5b37c4fcf6f780d41c15100b9a90b0c9f173e: Status 404 returned error can't find the container with id c8784fd857a21999d37199e98ce5b37c4fcf6f780d41c15100b9a90b0c9f173e
May 28 01:03:53.848975 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:53.848938    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" event={"ID":"e427384d-69e1-4cfc-b796-9f35349b668c","Type":"ContainerStarted","Data":"66d385dbbf4e0c050ce14e1a1e78c6bb175ba118cb9f705d890d17735692e77c"}
May 28 01:03:53.848975 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:53.848975    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" event={"ID":"e427384d-69e1-4cfc-b796-9f35349b668c","Type":"ContainerStarted","Data":"c8784fd857a21999d37199e98ce5b37c4fcf6f780d41c15100b9a90b0c9f173e"}
May 28 01:03:57.866080 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:57.865986    2550 generic.go:358] "Generic (PLEG): container finished" podID="e427384d-69e1-4cfc-b796-9f35349b668c" containerID="66d385dbbf4e0c050ce14e1a1e78c6bb175ba118cb9f705d890d17735692e77c" exitCode=0
May 28 01:03:57.866080 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:57.866046    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" event={"ID":"e427384d-69e1-4cfc-b796-9f35349b668c","Type":"ContainerDied","Data":"66d385dbbf4e0c050ce14e1a1e78c6bb175ba118cb9f705d890d17735692e77c"}
May 28 01:03:58.341841 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:58.341798    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.18:8443: connect: connection refused"
May 28 01:03:58.655749 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:58.655644    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.18:6567/health\": dial tcp 10.133.0.18:6567: connect: connection refused"
May 28 01:03:58.870775 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:03:58.870740    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" event={"ID":"e427384d-69e1-4cfc-b796-9f35349b668c","Type":"ContainerStarted","Data":"6bc803d178a275fbfd7865d9203d23bd79c093a17a3d5c7af24755388ac4df2f"}
May 28 01:04:03.886962 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:03.886927    2550 generic.go:358] "Generic (PLEG): container finished" podID="e427384d-69e1-4cfc-b796-9f35349b668c" containerID="6bc803d178a275fbfd7865d9203d23bd79c093a17a3d5c7af24755388ac4df2f" exitCode=0
May 28 01:04:03.887372 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:03.886975    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" event={"ID":"e427384d-69e1-4cfc-b796-9f35349b668c","Type":"ContainerDied","Data":"6bc803d178a275fbfd7865d9203d23bd79c093a17a3d5c7af24755388ac4df2f"}
May 28 01:04:04.894610 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:04.894567    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" event={"ID":"e427384d-69e1-4cfc-b796-9f35349b668c","Type":"ContainerStarted","Data":"8fc59d3017accc12aa48b9253a4186e5dc76904415894e2017535eb3d336b561"}
May 28 01:04:04.894610 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:04.894617    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" event={"ID":"e427384d-69e1-4cfc-b796-9f35349b668c","Type":"ContainerStarted","Data":"7dc5c7c2ecc6f680a07148c0d3af90228aad9007241216a6deb90c0e07de843a"}
May 28 01:04:04.895156 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:04.894634    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" event={"ID":"e427384d-69e1-4cfc-b796-9f35349b668c","Type":"ContainerStarted","Data":"96e85b236a57593d2a0b4729cd491e5df7fea9ea919e65b5aea418cc16ea32e2"}
May 28 01:04:04.895156 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:04.894648    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" event={"ID":"e427384d-69e1-4cfc-b796-9f35349b668c","Type":"ContainerStarted","Data":"ac3b389e1bb0f41fa45b474be10fe076b6946d04495b9188d76200ccd71fb0d4"}
May 28 01:04:04.919201 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:04.919123    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" podStartSLOduration=13.919103006 podStartE2EDuration="13.919103006s" podCreationTimestamp="2026-05-28 01:03:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 01:04:04.917126372 +0000 UTC m=+504.198083466" watchObservedRunningTime="2026-05-28 01:04:04.919103006 +0000 UTC m=+504.200060101"
May 28 01:04:07.867283 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:07.867032    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:04:07.867283 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:07.867084    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:04:07.867283 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:07.867101    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:04:07.867283 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:07.867225    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:04:07.869153 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:07.869118    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.20:8816: connect: connection refused"
May 28 01:04:07.869285 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:07.869151    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused"
May 28 01:04:07.869285 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:07.869206    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.20:6571: connect: connection refused"
May 28 01:04:07.869538 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:07.869506    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused"
May 28 01:04:08.342123 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:08.342073    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.18:8443: connect: connection refused"
May 28 01:04:08.655878 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:08.655382    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.18:6567/health\": dial tcp 10.133.0.18:6567: connect: connection refused"
May 28 01:04:10.868694 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:10.868655    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:04:10.869094 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:10.868828    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:04:10.869224 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:10.869200    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:04:10.869312 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:10.869239    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:04:10.869312 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:10.869256    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:04:10.869312 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:10.869270    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:04:10.869609 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:10.869591    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:04:10.869703 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:10.869685    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:04:10.869746 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:10.869733    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:04:10.873268 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:10.873250    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:04:10.921980 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:10.921949    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:04:10.924873 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:10.924851    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:04:18.342655 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:18.342609    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.18:8443: connect: connection refused"
May 28 01:04:18.343144 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:18.342745    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:04:18.655875 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:18.655786    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="online" probeResult="failure" output="Get \"https://10.133.0.18:6567/health\": dial tcp 10.133.0.18:6567: connect: connection refused"
May 28 01:04:18.656048 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:18.655926    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:04:21.957324 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:21.957233    2550 generic.go:358] "Generic (PLEG): container finished" podID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerID="cbb7ff6b5d2e237f3ced00d165fa1c9020c26b0275582cdfbdc898ecd097f733" exitCode=137
May 28 01:04:21.957324 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:21.957260    2550 generic.go:358] "Generic (PLEG): container finished" podID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerID="5d108bfc0052e6f7332bc2040f6b7686a99252fed456657e8337325bfb0795fa" exitCode=137
May 28 01:04:21.957324 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:21.957307    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" event={"ID":"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0","Type":"ContainerDied","Data":"cbb7ff6b5d2e237f3ced00d165fa1c9020c26b0275582cdfbdc898ecd097f733"}
May 28 01:04:21.957748 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:21.957340    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" event={"ID":"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0","Type":"ContainerDied","Data":"5d108bfc0052e6f7332bc2040f6b7686a99252fed456657e8337325bfb0795fa"}
May 28 01:04:22.332545 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.332523    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:04:22.494022 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.493941    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhz6n\" (UniqueName: \"kubernetes.io/projected/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-kube-api-access-vhz6n\") pod \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\" (UID: \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\") "
May 28 01:04:22.494022 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.493980    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-offline-tls\") pod \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\" (UID: \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\") "
May 28 01:04:22.494022 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.494009    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-feast-data\") pod \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\" (UID: \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\") "
May 28 01:04:22.494300 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.494073    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-ui-tls\") pod \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\" (UID: \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\") "
May 28 01:04:22.494300 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.494094    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-registry-tls\") pod \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\" (UID: \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\") "
May 28 01:04:22.494300 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.494120    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-online-tls\") pod \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\" (UID: \"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0\") "
May 28 01:04:22.494675 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.494639    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-feast-data" (OuterVolumeSpecName: "feast-data") pod "dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" (UID: "dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 01:04:22.496467 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.496431    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-online-tls" (OuterVolumeSpecName: "online-tls") pod "dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" (UID: "dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:04:22.496563 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.496457    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-kube-api-access-vhz6n" (OuterVolumeSpecName: "kube-api-access-vhz6n") pod "dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" (UID: "dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0"). InnerVolumeSpecName "kube-api-access-vhz6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 01:04:22.496563 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.496544    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" (UID: "dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:04:22.496636 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.496578    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" (UID: "dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:04:22.496636 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.496593    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" (UID: "dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:04:22.595337 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.595299    2550 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-ui-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:04:22.595337 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.595331    2550 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-registry-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:04:22.595337 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.595341    2550 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-online-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:04:22.595337 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.595349    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhz6n\" (UniqueName: \"kubernetes.io/projected/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-kube-api-access-vhz6n\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:04:22.595626 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.595358    2550 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-offline-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:04:22.595626 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.595367    2550 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0-feast-data\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:04:22.962467 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.962441    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"
May 28 01:04:22.962856 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.962440    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx" event={"ID":"dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0","Type":"ContainerDied","Data":"3f6b40d5997ef3f160214c83cf259e1e2f620f3604e53cab64b570039b1b319f"}
May 28 01:04:22.962856 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.962551    2550 scope.go:117] "RemoveContainer" containerID="cc345928d00e2109b6b1efbdfbedfbc59804b96023c6d4a9c1b6b6d7eda06c08"
May 28 01:04:22.970999 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.970982    2550 scope.go:117] "RemoveContainer" containerID="cbb7ff6b5d2e237f3ced00d165fa1c9020c26b0275582cdfbdc898ecd097f733"
May 28 01:04:22.978201 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.978186    2550 scope.go:117] "RemoveContainer" containerID="89111c246559d82e931d6a697bdbf223d73572012c409603d6edc40e5f6d1684"
May 28 01:04:22.983874 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.983851    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"]
May 28 01:04:22.985496 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.985478    2550 scope.go:117] "RemoveContainer" containerID="5d108bfc0052e6f7332bc2040f6b7686a99252fed456657e8337325bfb0795fa"
May 28 01:04:22.989260 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.989230    2550 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-mhwzx"]
May 28 01:04:22.993013 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.992997    2550 scope.go:117] "RemoveContainer" containerID="285f85cc5698b5217a12332c00df4cc13af7fe17f2f0ccb1be90eaac0f49986f"
May 28 01:04:22.999902 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:22.999881    2550 scope.go:117] "RemoveContainer" containerID="103f468642c988fbcf2c26e343b18e3507ce26c4cbe40dc1d8659cdab981912f"
May 28 01:04:23.323130 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:04:23.323059    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" path="/var/lib/kubelet/pods/dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0/volumes"
May 28 01:05:13.611652 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.611617    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"]
May 28 01:05:13.612163 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.611997    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="feast-apply"
May 28 01:05:13.612163 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.612011    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="feast-apply"
May 28 01:05:13.612163 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.612037    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="registry"
May 28 01:05:13.612163 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.612046    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="registry"
May 28 01:05:13.612163 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.612059    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="online"
May 28 01:05:13.612163 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.612068    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="online"
May 28 01:05:13.612163 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.612078    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="offline"
May 28 01:05:13.612163 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.612086    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="offline"
May 28 01:05:13.612163 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.612100    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="feast-init"
May 28 01:05:13.612163 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.612108    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="feast-init"
May 28 01:05:13.612163 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.612117    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="ui"
May 28 01:05:13.612163 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.612125    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="ui"
May 28 01:05:13.612577 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.612183    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="online"
May 28 01:05:13.612577 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.612199    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="ui"
May 28 01:05:13.612577 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.612211    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="offline"
May 28 01:05:13.612577 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.612222    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="dcd634b8-b6bf-46af-ad7c-d6c5bab5efd0" containerName="registry"
May 28 01:05:13.615668 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.615650    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:13.617942 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.617906    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 28 01:05:13.618075 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.617943    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 28 01:05:13.618075 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.617994    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 28 01:05:13.618075 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.617996    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 28 01:05:13.618075 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.618017    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-kwl75\""
May 28 01:05:13.618276 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.618233    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\""
May 28 01:05:13.618325 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.618276    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 28 01:05:13.618373 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.618352    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 28 01:05:13.626370 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.626335    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"]
May 28 01:05:13.723208 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.723171    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/c059fa85-5251-4ae1-8971-1db4c919e650-registry-tls\") pod \"feast-simple-feast-remote-setup-6454ffcbc-plzh5\" (UID: \"c059fa85-5251-4ae1-8971-1db4c919e650\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:13.723208 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.723208    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c059fa85-5251-4ae1-8971-1db4c919e650-feast-data\") pod \"feast-simple-feast-remote-setup-6454ffcbc-plzh5\" (UID: \"c059fa85-5251-4ae1-8971-1db4c919e650\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:13.723472 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.723240    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c059fa85-5251-4ae1-8971-1db4c919e650-offline-tls\") pod \"feast-simple-feast-remote-setup-6454ffcbc-plzh5\" (UID: \"c059fa85-5251-4ae1-8971-1db4c919e650\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:13.723472 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.723314    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t5rw\" (UniqueName: \"kubernetes.io/projected/c059fa85-5251-4ae1-8971-1db4c919e650-kube-api-access-7t5rw\") pod \"feast-simple-feast-remote-setup-6454ffcbc-plzh5\" (UID: \"c059fa85-5251-4ae1-8971-1db4c919e650\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:13.723472 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.723348    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c059fa85-5251-4ae1-8971-1db4c919e650-online-tls\") pod \"feast-simple-feast-remote-setup-6454ffcbc-plzh5\" (UID: \"c059fa85-5251-4ae1-8971-1db4c919e650\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:13.723472 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.723387    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c059fa85-5251-4ae1-8971-1db4c919e650-ui-tls\") pod \"feast-simple-feast-remote-setup-6454ffcbc-plzh5\" (UID: \"c059fa85-5251-4ae1-8971-1db4c919e650\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:13.823941 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.823905    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c059fa85-5251-4ae1-8971-1db4c919e650-offline-tls\") pod \"feast-simple-feast-remote-setup-6454ffcbc-plzh5\" (UID: \"c059fa85-5251-4ae1-8971-1db4c919e650\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:13.824104 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.823950    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7t5rw\" (UniqueName: \"kubernetes.io/projected/c059fa85-5251-4ae1-8971-1db4c919e650-kube-api-access-7t5rw\") pod \"feast-simple-feast-remote-setup-6454ffcbc-plzh5\" (UID: \"c059fa85-5251-4ae1-8971-1db4c919e650\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:13.824104 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:05:13.824048    2550 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found
May 28 01:05:13.824104 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.824066    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c059fa85-5251-4ae1-8971-1db4c919e650-online-tls\") pod \"feast-simple-feast-remote-setup-6454ffcbc-plzh5\" (UID: \"c059fa85-5251-4ae1-8971-1db4c919e650\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:13.824243 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:05:13.824119    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c059fa85-5251-4ae1-8971-1db4c919e650-offline-tls podName:c059fa85-5251-4ae1-8971-1db4c919e650 nodeName:}" failed. No retries permitted until 2026-05-28 01:05:14.32410147 +0000 UTC m=+573.605058560 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/c059fa85-5251-4ae1-8971-1db4c919e650-offline-tls") pod "feast-simple-feast-remote-setup-6454ffcbc-plzh5" (UID: "c059fa85-5251-4ae1-8971-1db4c919e650") : secret "feast-simple-feast-remote-setup-offline-tls" not found
May 28 01:05:13.824243 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.824148    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c059fa85-5251-4ae1-8971-1db4c919e650-ui-tls\") pod \"feast-simple-feast-remote-setup-6454ffcbc-plzh5\" (UID: \"c059fa85-5251-4ae1-8971-1db4c919e650\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:13.824357 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.824241    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/c059fa85-5251-4ae1-8971-1db4c919e650-registry-tls\") pod \"feast-simple-feast-remote-setup-6454ffcbc-plzh5\" (UID: \"c059fa85-5251-4ae1-8971-1db4c919e650\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:13.824357 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.824272    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c059fa85-5251-4ae1-8971-1db4c919e650-feast-data\") pod \"feast-simple-feast-remote-setup-6454ffcbc-plzh5\" (UID: \"c059fa85-5251-4ae1-8971-1db4c919e650\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:13.824689 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.824661    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c059fa85-5251-4ae1-8971-1db4c919e650-feast-data\") pod \"feast-simple-feast-remote-setup-6454ffcbc-plzh5\" (UID: \"c059fa85-5251-4ae1-8971-1db4c919e650\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:13.824884 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.824866    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/c059fa85-5251-4ae1-8971-1db4c919e650-registry-tls\") pod \"feast-simple-feast-remote-setup-6454ffcbc-plzh5\" (UID: \"c059fa85-5251-4ae1-8971-1db4c919e650\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:13.826557 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.826539    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c059fa85-5251-4ae1-8971-1db4c919e650-online-tls\") pod \"feast-simple-feast-remote-setup-6454ffcbc-plzh5\" (UID: \"c059fa85-5251-4ae1-8971-1db4c919e650\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:13.826633 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.826588    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c059fa85-5251-4ae1-8971-1db4c919e650-ui-tls\") pod \"feast-simple-feast-remote-setup-6454ffcbc-plzh5\" (UID: \"c059fa85-5251-4ae1-8971-1db4c919e650\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:13.836277 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:13.836254    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t5rw\" (UniqueName: \"kubernetes.io/projected/c059fa85-5251-4ae1-8971-1db4c919e650-kube-api-access-7t5rw\") pod \"feast-simple-feast-remote-setup-6454ffcbc-plzh5\" (UID: \"c059fa85-5251-4ae1-8971-1db4c919e650\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:14.328992 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:14.328959    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c059fa85-5251-4ae1-8971-1db4c919e650-offline-tls\") pod \"feast-simple-feast-remote-setup-6454ffcbc-plzh5\" (UID: \"c059fa85-5251-4ae1-8971-1db4c919e650\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:14.331372 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:14.331348    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c059fa85-5251-4ae1-8971-1db4c919e650-offline-tls\") pod \"feast-simple-feast-remote-setup-6454ffcbc-plzh5\" (UID: \"c059fa85-5251-4ae1-8971-1db4c919e650\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:14.525659 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:14.525614    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:14.664302 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:14.664274    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"]
May 28 01:05:14.666325 ip-10-0-131-73 kubenswrapper[2550]: W0528 01:05:14.666296    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc059fa85_5251_4ae1_8971_1db4c919e650.slice/crio-2a3e1bdbd9b1560945b18b8226692ab3fcd3cb9148137ab4eec5d6f51159e077 WatchSource:0}: Error finding container 2a3e1bdbd9b1560945b18b8226692ab3fcd3cb9148137ab4eec5d6f51159e077: Status 404 returned error can't find the container with id 2a3e1bdbd9b1560945b18b8226692ab3fcd3cb9148137ab4eec5d6f51159e077
May 28 01:05:15.108836 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:15.108796    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5" event={"ID":"c059fa85-5251-4ae1-8971-1db4c919e650","Type":"ContainerStarted","Data":"a622bf8a798eb74ecc840153679ad1db206c7e90b4dd94a12abc219363a131f1"}
May 28 01:05:15.108836 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:15.108837    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5" event={"ID":"c059fa85-5251-4ae1-8971-1db4c919e650","Type":"ContainerStarted","Data":"2a3e1bdbd9b1560945b18b8226692ab3fcd3cb9148137ab4eec5d6f51159e077"}
May 28 01:05:20.123980 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:20.123946    2550 generic.go:358] "Generic (PLEG): container finished" podID="c059fa85-5251-4ae1-8971-1db4c919e650" containerID="a622bf8a798eb74ecc840153679ad1db206c7e90b4dd94a12abc219363a131f1" exitCode=0
May 28 01:05:20.124520 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:20.124025    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5" event={"ID":"c059fa85-5251-4ae1-8971-1db4c919e650","Type":"ContainerDied","Data":"a622bf8a798eb74ecc840153679ad1db206c7e90b4dd94a12abc219363a131f1"}
May 28 01:05:21.128688 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:21.128657    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5" event={"ID":"c059fa85-5251-4ae1-8971-1db4c919e650","Type":"ContainerStarted","Data":"eecc040c7c0f6943baa722e0ee67b75fb18b048df2d976ee098581c67797c5d2"}
May 28 01:05:26.143510 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:26.143477    2550 generic.go:358] "Generic (PLEG): container finished" podID="c059fa85-5251-4ae1-8971-1db4c919e650" containerID="eecc040c7c0f6943baa722e0ee67b75fb18b048df2d976ee098581c67797c5d2" exitCode=0
May 28 01:05:26.143875 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:26.143550    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5" event={"ID":"c059fa85-5251-4ae1-8971-1db4c919e650","Type":"ContainerDied","Data":"eecc040c7c0f6943baa722e0ee67b75fb18b048df2d976ee098581c67797c5d2"}
May 28 01:05:27.149793 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:27.149747    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5" event={"ID":"c059fa85-5251-4ae1-8971-1db4c919e650","Type":"ContainerStarted","Data":"e8dbb30f43f4b4c9e27f1c12b1ec2695f8a8d1ecf54d5d1e7f002bd14c186b7a"}
May 28 01:05:27.150275 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:27.149801    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5" event={"ID":"c059fa85-5251-4ae1-8971-1db4c919e650","Type":"ContainerStarted","Data":"14dee41942668462658afcc0333e5b88c97a97188448d52d5ea429c09f181fab"}
May 28 01:05:27.150275 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:27.149817    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5" event={"ID":"c059fa85-5251-4ae1-8971-1db4c919e650","Type":"ContainerStarted","Data":"b99affd5344c21f0b2d940ddd844900c51dd9d8db16d5d64671080827db07530"}
May 28 01:05:27.172204 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:27.172096    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5" podStartSLOduration=14.172075169 podStartE2EDuration="14.172075169s" podCreationTimestamp="2026-05-28 01:05:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 01:05:27.171508541 +0000 UTC m=+586.452465636" watchObservedRunningTime="2026-05-28 01:05:27.172075169 +0000 UTC m=+586.453032262"
May 28 01:05:29.525808 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:29.525702    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:29.525808 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:29.525766    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:29.525808 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:29.525782    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:29.527907 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:29.527843    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5" podUID="c059fa85-5251-4ae1-8971-1db4c919e650" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.21:8443: connect: connection refused"
May 28 01:05:29.527907 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:29.527874    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5" podUID="c059fa85-5251-4ae1-8971-1db4c919e650" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.21:8816: connect: connection refused"
May 28 01:05:29.527907 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:29.527898    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5" podUID="c059fa85-5251-4ae1-8971-1db4c919e650" containerName="online" probeResult="failure" output="Get \"https://10.133.0.21:6567/health\": dial tcp 10.133.0.21:6567: connect: connection refused"
May 28 01:05:32.527022 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:32.526992    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:32.527528 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:32.527110    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:32.527528 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:32.527445    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:32.527528 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:32.527475    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:32.527744 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:32.527722    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:32.527916 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:32.527900    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:32.531139 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:32.531119    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:33.168710 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:33.168679    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:33.171531 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:33.171511    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:05:41.249820 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:41.249793    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9z98m_6671c1c1-b674-4a1d-9011-1ca22724b234/ovn-acl-logging/0.log"
May 28 01:05:41.250254 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:05:41.249798    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9z98m_6671c1c1-b674-4a1d-9011-1ca22724b234/ovn-acl-logging/0.log"
May 28 01:06:34.814733 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:06:34.814698    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"]
May 28 01:06:34.817375 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:06:34.815105    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5" podUID="c059fa85-5251-4ae1-8971-1db4c919e650" containerName="online" containerID="cri-o://b99affd5344c21f0b2d940ddd844900c51dd9d8db16d5d64671080827db07530" gracePeriod=30
May 28 01:06:34.817375 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:06:34.815236    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5" podUID="c059fa85-5251-4ae1-8971-1db4c919e650" containerName="ui" containerID="cri-o://e8dbb30f43f4b4c9e27f1c12b1ec2695f8a8d1ecf54d5d1e7f002bd14c186b7a" gracePeriod=30
May 28 01:06:34.817375 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:06:34.815297    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5" podUID="c059fa85-5251-4ae1-8971-1db4c919e650" containerName="offline" containerID="cri-o://14dee41942668462658afcc0333e5b88c97a97188448d52d5ea429c09f181fab" gracePeriod=30
May 28 01:06:34.927057 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:06:34.927017    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"]
May 28 01:06:34.927404 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:06:34.927376    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="registry" containerID="cri-o://ac3b389e1bb0f41fa45b474be10fe076b6946d04495b9188d76200ccd71fb0d4" gracePeriod=30
May 28 01:06:34.927680 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:06:34.927464    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="online" containerID="cri-o://96e85b236a57593d2a0b4729cd491e5df7fea9ea919e65b5aea418cc16ea32e2" gracePeriod=30
May 28 01:06:34.927823 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:06:34.927468    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="ui" containerID="cri-o://8fc59d3017accc12aa48b9253a4186e5dc76904415894e2017535eb3d336b561" gracePeriod=30
May 28 01:06:34.927823 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:06:34.927482    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="offline" containerID="cri-o://7dc5c7c2ecc6f680a07148c0d3af90228aad9007241216a6deb90c0e07de843a" gracePeriod=30
May 28 01:06:35.348715 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:06:35.348681    2550 generic.go:358] "Generic (PLEG): container finished" podID="c059fa85-5251-4ae1-8971-1db4c919e650" containerID="e8dbb30f43f4b4c9e27f1c12b1ec2695f8a8d1ecf54d5d1e7f002bd14c186b7a" exitCode=0
May 28 01:06:35.348880 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:06:35.348748    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5" event={"ID":"c059fa85-5251-4ae1-8971-1db4c919e650","Type":"ContainerDied","Data":"e8dbb30f43f4b4c9e27f1c12b1ec2695f8a8d1ecf54d5d1e7f002bd14c186b7a"}
May 28 01:06:36.353406 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:06:36.353373    2550 generic.go:358] "Generic (PLEG): container finished" podID="c059fa85-5251-4ae1-8971-1db4c919e650" containerID="b99affd5344c21f0b2d940ddd844900c51dd9d8db16d5d64671080827db07530" exitCode=0
May 28 01:06:36.353844 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:06:36.353445    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5" event={"ID":"c059fa85-5251-4ae1-8971-1db4c919e650","Type":"ContainerDied","Data":"b99affd5344c21f0b2d940ddd844900c51dd9d8db16d5d64671080827db07530"}
May 28 01:06:36.355854 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:06:36.355831    2550 generic.go:358] "Generic (PLEG): container finished" podID="e427384d-69e1-4cfc-b796-9f35349b668c" containerID="8fc59d3017accc12aa48b9253a4186e5dc76904415894e2017535eb3d336b561" exitCode=0
May 28 01:06:36.355854 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:06:36.355849    2550 generic.go:358] "Generic (PLEG): container finished" podID="e427384d-69e1-4cfc-b796-9f35349b668c" containerID="96e85b236a57593d2a0b4729cd491e5df7fea9ea919e65b5aea418cc16ea32e2" exitCode=0
May 28 01:06:36.356008 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:06:36.355879    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" event={"ID":"e427384d-69e1-4cfc-b796-9f35349b668c","Type":"ContainerDied","Data":"8fc59d3017accc12aa48b9253a4186e5dc76904415894e2017535eb3d336b561"}
May 28 01:06:36.356008 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:06:36.355898    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" event={"ID":"e427384d-69e1-4cfc-b796-9f35349b668c","Type":"ContainerDied","Data":"96e85b236a57593d2a0b4729cd491e5df7fea9ea919e65b5aea418cc16ea32e2"}
May 28 01:06:40.869746 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:06:40.869689    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused"
May 28 01:06:40.922833 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:06:40.922792    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused"
May 28 01:06:42.528099 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:06:42.528055    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5" podUID="c059fa85-5251-4ae1-8971-1db4c919e650" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.21:8443: connect: connection refused"
May 28 01:06:43.169096 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:06:43.169042    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5" podUID="c059fa85-5251-4ae1-8971-1db4c919e650" containerName="online" probeResult="failure" output="Get \"https://10.133.0.21:6567/health\": dial tcp 10.133.0.21:6567: connect: connection refused"
May 28 01:06:50.869683 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:06:50.869639    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused"
May 28 01:06:50.923102 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:06:50.923040    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused"
May 28 01:06:52.528102 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:06:52.528050    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5" podUID="c059fa85-5251-4ae1-8971-1db4c919e650" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.21:8443: connect: connection refused"
May 28 01:06:53.169919 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:06:53.169874    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5" podUID="c059fa85-5251-4ae1-8971-1db4c919e650" containerName="online" probeResult="failure" output="Get \"https://10.133.0.21:6567/health\": dial tcp 10.133.0.21:6567: connect: connection refused"
May 28 01:07:00.870229 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:00.870176    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused"
May 28 01:07:00.870703 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:00.870334    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:07:00.922431 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:00.922364    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused"
May 28 01:07:00.922609 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:00.922525    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:07:02.527971 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:02.527931    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5" podUID="c059fa85-5251-4ae1-8971-1db4c919e650" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.21:8443: connect: connection refused"
May 28 01:07:02.528366 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:02.528077    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:07:03.169834 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:03.169781    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5" podUID="c059fa85-5251-4ae1-8971-1db4c919e650" containerName="online" probeResult="failure" output="Get \"https://10.133.0.21:6567/health\": dial tcp 10.133.0.21:6567: connect: connection refused"
May 28 01:07:03.170007 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:03.169948    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:07:05.045930 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.045906    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:07:05.173135 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.173102    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c059fa85-5251-4ae1-8971-1db4c919e650-ui-tls\") pod \"c059fa85-5251-4ae1-8971-1db4c919e650\" (UID: \"c059fa85-5251-4ae1-8971-1db4c919e650\") "
May 28 01:07:05.173327 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.173157    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c059fa85-5251-4ae1-8971-1db4c919e650-feast-data\") pod \"c059fa85-5251-4ae1-8971-1db4c919e650\" (UID: \"c059fa85-5251-4ae1-8971-1db4c919e650\") "
May 28 01:07:05.173327 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.173201    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/c059fa85-5251-4ae1-8971-1db4c919e650-registry-tls\") pod \"c059fa85-5251-4ae1-8971-1db4c919e650\" (UID: \"c059fa85-5251-4ae1-8971-1db4c919e650\") "
May 28 01:07:05.173327 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.173225    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c059fa85-5251-4ae1-8971-1db4c919e650-offline-tls\") pod \"c059fa85-5251-4ae1-8971-1db4c919e650\" (UID: \"c059fa85-5251-4ae1-8971-1db4c919e650\") "
May 28 01:07:05.173327 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.173257    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7t5rw\" (UniqueName: \"kubernetes.io/projected/c059fa85-5251-4ae1-8971-1db4c919e650-kube-api-access-7t5rw\") pod \"c059fa85-5251-4ae1-8971-1db4c919e650\" (UID: \"c059fa85-5251-4ae1-8971-1db4c919e650\") "
May 28 01:07:05.173327 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.173286    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c059fa85-5251-4ae1-8971-1db4c919e650-online-tls\") pod \"c059fa85-5251-4ae1-8971-1db4c919e650\" (UID: \"c059fa85-5251-4ae1-8971-1db4c919e650\") "
May 28 01:07:05.173668 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.173627    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c059fa85-5251-4ae1-8971-1db4c919e650-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c059fa85-5251-4ae1-8971-1db4c919e650" (UID: "c059fa85-5251-4ae1-8971-1db4c919e650"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 01:07:05.173792 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.173761    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c059fa85-5251-4ae1-8971-1db4c919e650-feast-data" (OuterVolumeSpecName: "feast-data") pod "c059fa85-5251-4ae1-8971-1db4c919e650" (UID: "c059fa85-5251-4ae1-8971-1db4c919e650"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 01:07:05.175512 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.175477    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c059fa85-5251-4ae1-8971-1db4c919e650-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "c059fa85-5251-4ae1-8971-1db4c919e650" (UID: "c059fa85-5251-4ae1-8971-1db4c919e650"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:07:05.175614 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.175513    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c059fa85-5251-4ae1-8971-1db4c919e650-online-tls" (OuterVolumeSpecName: "online-tls") pod "c059fa85-5251-4ae1-8971-1db4c919e650" (UID: "c059fa85-5251-4ae1-8971-1db4c919e650"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:07:05.175815 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.175793    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c059fa85-5251-4ae1-8971-1db4c919e650-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "c059fa85-5251-4ae1-8971-1db4c919e650" (UID: "c059fa85-5251-4ae1-8971-1db4c919e650"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:07:05.175867 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.175822    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c059fa85-5251-4ae1-8971-1db4c919e650-kube-api-access-7t5rw" (OuterVolumeSpecName: "kube-api-access-7t5rw") pod "c059fa85-5251-4ae1-8971-1db4c919e650" (UID: "c059fa85-5251-4ae1-8971-1db4c919e650"). InnerVolumeSpecName "kube-api-access-7t5rw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 01:07:05.274564 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.274517    2550 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c059fa85-5251-4ae1-8971-1db4c919e650-ui-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:07:05.274564 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.274559    2550 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c059fa85-5251-4ae1-8971-1db4c919e650-feast-data\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:07:05.274564 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.274570    2550 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/c059fa85-5251-4ae1-8971-1db4c919e650-registry-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:07:05.274564 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.274580    2550 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c059fa85-5251-4ae1-8971-1db4c919e650-offline-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:07:05.274830 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.274588    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7t5rw\" (UniqueName: \"kubernetes.io/projected/c059fa85-5251-4ae1-8971-1db4c919e650-kube-api-access-7t5rw\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:07:05.274830 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.274597    2550 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c059fa85-5251-4ae1-8971-1db4c919e650-online-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:07:05.444987 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.444898    2550 generic.go:358] "Generic (PLEG): container finished" podID="e427384d-69e1-4cfc-b796-9f35349b668c" containerID="7dc5c7c2ecc6f680a07148c0d3af90228aad9007241216a6deb90c0e07de843a" exitCode=137
May 28 01:07:05.444987 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.444928    2550 generic.go:358] "Generic (PLEG): container finished" podID="e427384d-69e1-4cfc-b796-9f35349b668c" containerID="ac3b389e1bb0f41fa45b474be10fe076b6946d04495b9188d76200ccd71fb0d4" exitCode=137
May 28 01:07:05.444987 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.444968    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" event={"ID":"e427384d-69e1-4cfc-b796-9f35349b668c","Type":"ContainerDied","Data":"7dc5c7c2ecc6f680a07148c0d3af90228aad9007241216a6deb90c0e07de843a"}
May 28 01:07:05.445183 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.445001    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" event={"ID":"e427384d-69e1-4cfc-b796-9f35349b668c","Type":"ContainerDied","Data":"ac3b389e1bb0f41fa45b474be10fe076b6946d04495b9188d76200ccd71fb0d4"}
May 28 01:07:05.447157 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.447077    2550 generic.go:358] "Generic (PLEG): container finished" podID="c059fa85-5251-4ae1-8971-1db4c919e650" containerID="14dee41942668462658afcc0333e5b88c97a97188448d52d5ea429c09f181fab" exitCode=137
May 28 01:07:05.447157 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.447132    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5" event={"ID":"c059fa85-5251-4ae1-8971-1db4c919e650","Type":"ContainerDied","Data":"14dee41942668462658afcc0333e5b88c97a97188448d52d5ea429c09f181fab"}
May 28 01:07:05.447157 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.447151    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5" event={"ID":"c059fa85-5251-4ae1-8971-1db4c919e650","Type":"ContainerDied","Data":"2a3e1bdbd9b1560945b18b8226692ab3fcd3cb9148137ab4eec5d6f51159e077"}
May 28 01:07:05.447281 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.447159    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"
May 28 01:07:05.447281 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.447166    2550 scope.go:117] "RemoveContainer" containerID="e8dbb30f43f4b4c9e27f1c12b1ec2695f8a8d1ecf54d5d1e7f002bd14c186b7a"
May 28 01:07:05.455641 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.455624    2550 scope.go:117] "RemoveContainer" containerID="14dee41942668462658afcc0333e5b88c97a97188448d52d5ea429c09f181fab"
May 28 01:07:05.463489 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.463456    2550 scope.go:117] "RemoveContainer" containerID="b99affd5344c21f0b2d940ddd844900c51dd9d8db16d5d64671080827db07530"
May 28 01:07:05.466664 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.466634    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"]
May 28 01:07:05.472165 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.472140    2550 scope.go:117] "RemoveContainer" containerID="eecc040c7c0f6943baa722e0ee67b75fb18b048df2d976ee098581c67797c5d2"
May 28 01:07:05.472666 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.472642    2550 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6454ffcbc-plzh5"]
May 28 01:07:05.481341 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.481319    2550 scope.go:117] "RemoveContainer" containerID="a622bf8a798eb74ecc840153679ad1db206c7e90b4dd94a12abc219363a131f1"
May 28 01:07:05.489038 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.489014    2550 scope.go:117] "RemoveContainer" containerID="e8dbb30f43f4b4c9e27f1c12b1ec2695f8a8d1ecf54d5d1e7f002bd14c186b7a"
May 28 01:07:05.489364 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:07:05.489336    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8dbb30f43f4b4c9e27f1c12b1ec2695f8a8d1ecf54d5d1e7f002bd14c186b7a\": container with ID starting with e8dbb30f43f4b4c9e27f1c12b1ec2695f8a8d1ecf54d5d1e7f002bd14c186b7a not found: ID does not exist" containerID="e8dbb30f43f4b4c9e27f1c12b1ec2695f8a8d1ecf54d5d1e7f002bd14c186b7a"
May 28 01:07:05.489447 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.489377    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8dbb30f43f4b4c9e27f1c12b1ec2695f8a8d1ecf54d5d1e7f002bd14c186b7a"} err="failed to get container status \"e8dbb30f43f4b4c9e27f1c12b1ec2695f8a8d1ecf54d5d1e7f002bd14c186b7a\": rpc error: code = NotFound desc = could not find container \"e8dbb30f43f4b4c9e27f1c12b1ec2695f8a8d1ecf54d5d1e7f002bd14c186b7a\": container with ID starting with e8dbb30f43f4b4c9e27f1c12b1ec2695f8a8d1ecf54d5d1e7f002bd14c186b7a not found: ID does not exist"
May 28 01:07:05.489447 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.489405    2550 scope.go:117] "RemoveContainer" containerID="14dee41942668462658afcc0333e5b88c97a97188448d52d5ea429c09f181fab"
May 28 01:07:05.489711 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:07:05.489688    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14dee41942668462658afcc0333e5b88c97a97188448d52d5ea429c09f181fab\": container with ID starting with 14dee41942668462658afcc0333e5b88c97a97188448d52d5ea429c09f181fab not found: ID does not exist" containerID="14dee41942668462658afcc0333e5b88c97a97188448d52d5ea429c09f181fab"
May 28 01:07:05.489766 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.489721    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14dee41942668462658afcc0333e5b88c97a97188448d52d5ea429c09f181fab"} err="failed to get container status \"14dee41942668462658afcc0333e5b88c97a97188448d52d5ea429c09f181fab\": rpc error: code = NotFound desc = could not find container \"14dee41942668462658afcc0333e5b88c97a97188448d52d5ea429c09f181fab\": container with ID starting with 14dee41942668462658afcc0333e5b88c97a97188448d52d5ea429c09f181fab not found: ID does not exist"
May 28 01:07:05.489766 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.489743    2550 scope.go:117] "RemoveContainer" containerID="b99affd5344c21f0b2d940ddd844900c51dd9d8db16d5d64671080827db07530"
May 28 01:07:05.489979 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:07:05.489962    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b99affd5344c21f0b2d940ddd844900c51dd9d8db16d5d64671080827db07530\": container with ID starting with b99affd5344c21f0b2d940ddd844900c51dd9d8db16d5d64671080827db07530 not found: ID does not exist" containerID="b99affd5344c21f0b2d940ddd844900c51dd9d8db16d5d64671080827db07530"
May 28 01:07:05.490023 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.489986    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b99affd5344c21f0b2d940ddd844900c51dd9d8db16d5d64671080827db07530"} err="failed to get container status \"b99affd5344c21f0b2d940ddd844900c51dd9d8db16d5d64671080827db07530\": rpc error: code = NotFound desc = could not find container \"b99affd5344c21f0b2d940ddd844900c51dd9d8db16d5d64671080827db07530\": container with ID starting with b99affd5344c21f0b2d940ddd844900c51dd9d8db16d5d64671080827db07530 not found: ID does not exist"
May 28 01:07:05.490023 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.490004    2550 scope.go:117] "RemoveContainer" containerID="eecc040c7c0f6943baa722e0ee67b75fb18b048df2d976ee098581c67797c5d2"
May 28 01:07:05.490303 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:07:05.490281    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eecc040c7c0f6943baa722e0ee67b75fb18b048df2d976ee098581c67797c5d2\": container with ID starting with eecc040c7c0f6943baa722e0ee67b75fb18b048df2d976ee098581c67797c5d2 not found: ID does not exist" containerID="eecc040c7c0f6943baa722e0ee67b75fb18b048df2d976ee098581c67797c5d2"
May 28 01:07:05.490365 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.490307    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eecc040c7c0f6943baa722e0ee67b75fb18b048df2d976ee098581c67797c5d2"} err="failed to get container status \"eecc040c7c0f6943baa722e0ee67b75fb18b048df2d976ee098581c67797c5d2\": rpc error: code = NotFound desc = could not find container \"eecc040c7c0f6943baa722e0ee67b75fb18b048df2d976ee098581c67797c5d2\": container with ID starting with eecc040c7c0f6943baa722e0ee67b75fb18b048df2d976ee098581c67797c5d2 not found: ID does not exist"
May 28 01:07:05.490365 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.490320    2550 scope.go:117] "RemoveContainer" containerID="a622bf8a798eb74ecc840153679ad1db206c7e90b4dd94a12abc219363a131f1"
May 28 01:07:05.490625 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:07:05.490607    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a622bf8a798eb74ecc840153679ad1db206c7e90b4dd94a12abc219363a131f1\": container with ID starting with a622bf8a798eb74ecc840153679ad1db206c7e90b4dd94a12abc219363a131f1 not found: ID does not exist" containerID="a622bf8a798eb74ecc840153679ad1db206c7e90b4dd94a12abc219363a131f1"
May 28 01:07:05.490674 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.490628    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a622bf8a798eb74ecc840153679ad1db206c7e90b4dd94a12abc219363a131f1"} err="failed to get container status \"a622bf8a798eb74ecc840153679ad1db206c7e90b4dd94a12abc219363a131f1\": rpc error: code = NotFound desc = could not find container \"a622bf8a798eb74ecc840153679ad1db206c7e90b4dd94a12abc219363a131f1\": container with ID starting with a622bf8a798eb74ecc840153679ad1db206c7e90b4dd94a12abc219363a131f1 not found: ID does not exist"
May 28 01:07:05.581877 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.581855    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:07:05.678643 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.678603    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-registry-tls\") pod \"e427384d-69e1-4cfc-b796-9f35349b668c\" (UID: \"e427384d-69e1-4cfc-b796-9f35349b668c\") "
May 28 01:07:05.678815 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.678667    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vj575\" (UniqueName: \"kubernetes.io/projected/e427384d-69e1-4cfc-b796-9f35349b668c-kube-api-access-vj575\") pod \"e427384d-69e1-4cfc-b796-9f35349b668c\" (UID: \"e427384d-69e1-4cfc-b796-9f35349b668c\") "
May 28 01:07:05.678815 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.678701    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-ui-tls\") pod \"e427384d-69e1-4cfc-b796-9f35349b668c\" (UID: \"e427384d-69e1-4cfc-b796-9f35349b668c\") "
May 28 01:07:05.678815 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.678727    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-offline-tls\") pod \"e427384d-69e1-4cfc-b796-9f35349b668c\" (UID: \"e427384d-69e1-4cfc-b796-9f35349b668c\") "
May 28 01:07:05.678815 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.678765    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e427384d-69e1-4cfc-b796-9f35349b668c-feast-data\") pod \"e427384d-69e1-4cfc-b796-9f35349b668c\" (UID: \"e427384d-69e1-4cfc-b796-9f35349b668c\") "
May 28 01:07:05.678815 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.678796    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-online-tls\") pod \"e427384d-69e1-4cfc-b796-9f35349b668c\" (UID: \"e427384d-69e1-4cfc-b796-9f35349b668c\") "
May 28 01:07:05.679395 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.679334    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e427384d-69e1-4cfc-b796-9f35349b668c-feast-data" (OuterVolumeSpecName: "feast-data") pod "e427384d-69e1-4cfc-b796-9f35349b668c" (UID: "e427384d-69e1-4cfc-b796-9f35349b668c"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 01:07:05.680938 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.680913    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "e427384d-69e1-4cfc-b796-9f35349b668c" (UID: "e427384d-69e1-4cfc-b796-9f35349b668c"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:07:05.681030 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.680973    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "e427384d-69e1-4cfc-b796-9f35349b668c" (UID: "e427384d-69e1-4cfc-b796-9f35349b668c"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:07:05.681030 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.680986    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e427384d-69e1-4cfc-b796-9f35349b668c-kube-api-access-vj575" (OuterVolumeSpecName: "kube-api-access-vj575") pod "e427384d-69e1-4cfc-b796-9f35349b668c" (UID: "e427384d-69e1-4cfc-b796-9f35349b668c"). InnerVolumeSpecName "kube-api-access-vj575". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 01:07:05.681208 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.681195    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e427384d-69e1-4cfc-b796-9f35349b668c" (UID: "e427384d-69e1-4cfc-b796-9f35349b668c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:07:05.681247 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.681212    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-online-tls" (OuterVolumeSpecName: "online-tls") pod "e427384d-69e1-4cfc-b796-9f35349b668c" (UID: "e427384d-69e1-4cfc-b796-9f35349b668c"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:07:05.779669 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.779576    2550 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-online-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:07:05.779669 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.779618    2550 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-registry-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:07:05.779669 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.779627    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vj575\" (UniqueName: \"kubernetes.io/projected/e427384d-69e1-4cfc-b796-9f35349b668c-kube-api-access-vj575\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:07:05.779669 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.779636    2550 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-ui-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:07:05.779669 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.779646    2550 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e427384d-69e1-4cfc-b796-9f35349b668c-offline-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:07:05.779669 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:05.779655    2550 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e427384d-69e1-4cfc-b796-9f35349b668c-feast-data\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:07:06.452711 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:06.452680    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df" event={"ID":"e427384d-69e1-4cfc-b796-9f35349b668c","Type":"ContainerDied","Data":"c8784fd857a21999d37199e98ce5b37c4fcf6f780d41c15100b9a90b0c9f173e"}
May 28 01:07:06.452711 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:06.452710    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"
May 28 01:07:06.453210 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:06.452728    2550 scope.go:117] "RemoveContainer" containerID="8fc59d3017accc12aa48b9253a4186e5dc76904415894e2017535eb3d336b561"
May 28 01:07:06.460768 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:06.460747    2550 scope.go:117] "RemoveContainer" containerID="7dc5c7c2ecc6f680a07148c0d3af90228aad9007241216a6deb90c0e07de843a"
May 28 01:07:06.468341 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:06.468320    2550 scope.go:117] "RemoveContainer" containerID="96e85b236a57593d2a0b4729cd491e5df7fea9ea919e65b5aea418cc16ea32e2"
May 28 01:07:06.475286 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:06.475268    2550 scope.go:117] "RemoveContainer" containerID="ac3b389e1bb0f41fa45b474be10fe076b6946d04495b9188d76200ccd71fb0d4"
May 28 01:07:06.482206 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:06.482188    2550 scope.go:117] "RemoveContainer" containerID="6bc803d178a275fbfd7865d9203d23bd79c093a17a3d5c7af24755388ac4df2f"
May 28 01:07:06.489143 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:06.489106    2550 scope.go:117] "RemoveContainer" containerID="66d385dbbf4e0c050ce14e1a1e78c6bb175ba118cb9f705d890d17735692e77c"
May 28 01:07:06.491187 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:06.491150    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"]
May 28 01:07:06.504067 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:06.504040    2550 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6ff8bdb7f4-755df"]
May 28 01:07:07.327877 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:07.324629    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c059fa85-5251-4ae1-8971-1db4c919e650" path="/var/lib/kubelet/pods/c059fa85-5251-4ae1-8971-1db4c919e650/volumes"
May 28 01:07:07.327877 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:07.325478    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" path="/var/lib/kubelet/pods/e427384d-69e1-4cfc-b796-9f35349b668c/volumes"
May 28 01:07:17.886762 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.886726    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-lcwn9"]
May 28 01:07:17.887200 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887089    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="feast-apply"
May 28 01:07:17.887200 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887107    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="feast-apply"
May 28 01:07:17.887200 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887126    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="feast-init"
May 28 01:07:17.887200 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887134    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="feast-init"
May 28 01:07:17.887200 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887144    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="ui"
May 28 01:07:17.887200 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887153    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="ui"
May 28 01:07:17.887200 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887163    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="registry"
May 28 01:07:17.887200 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887171    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="registry"
May 28 01:07:17.887200 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887180    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="offline"
May 28 01:07:17.887200 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887189    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="offline"
May 28 01:07:17.887200 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887203    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="online"
May 28 01:07:17.887602 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887211    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="online"
May 28 01:07:17.887602 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887219    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c059fa85-5251-4ae1-8971-1db4c919e650" containerName="feast-apply"
May 28 01:07:17.887602 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887227    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c059fa85-5251-4ae1-8971-1db4c919e650" containerName="feast-apply"
May 28 01:07:17.887602 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887236    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c059fa85-5251-4ae1-8971-1db4c919e650" containerName="offline"
May 28 01:07:17.887602 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887245    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c059fa85-5251-4ae1-8971-1db4c919e650" containerName="offline"
May 28 01:07:17.887602 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887259    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c059fa85-5251-4ae1-8971-1db4c919e650" containerName="online"
May 28 01:07:17.887602 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887267    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c059fa85-5251-4ae1-8971-1db4c919e650" containerName="online"
May 28 01:07:17.887602 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887279    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c059fa85-5251-4ae1-8971-1db4c919e650" containerName="ui"
May 28 01:07:17.887602 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887287    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c059fa85-5251-4ae1-8971-1db4c919e650" containerName="ui"
May 28 01:07:17.887602 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887303    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c059fa85-5251-4ae1-8971-1db4c919e650" containerName="feast-init"
May 28 01:07:17.887602 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887311    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c059fa85-5251-4ae1-8971-1db4c919e650" containerName="feast-init"
May 28 01:07:17.887602 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887366    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="registry"
May 28 01:07:17.887602 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887377    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="c059fa85-5251-4ae1-8971-1db4c919e650" containerName="offline"
May 28 01:07:17.887602 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887388    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="online"
May 28 01:07:17.887602 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887397    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="offline"
May 28 01:07:17.887602 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887406    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="c059fa85-5251-4ae1-8971-1db4c919e650" containerName="ui"
May 28 01:07:17.887602 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887434    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="e427384d-69e1-4cfc-b796-9f35349b668c" containerName="ui"
May 28 01:07:17.887602 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.887444    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="c059fa85-5251-4ae1-8971-1db4c919e650" containerName="online"
May 28 01:07:17.891598 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.891580    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-lcwn9"
May 28 01:07:17.893574 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.893548    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-8cz5l\""
May 28 01:07:17.893707 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.893588    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 28 01:07:17.894091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.894074    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\""
May 28 01:07:17.894153 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.894089    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 28 01:07:17.898470 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.898446    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-lcwn9"]
May 28 01:07:17.925540 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.925507    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-854544c579-xkw9w"]
May 28 01:07:17.928702 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.928677    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-854544c579-xkw9w"
May 28 01:07:17.934650 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.934622    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-854544c579-xkw9w"]
May 28 01:07:17.971786 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.971744    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/f310bbd7-973a-4449-a5fc-2bbab4aef6cd-postgresdata\") pod \"postgres-7c7656cc6d-lcwn9\" (UID: \"f310bbd7-973a-4449-a5fc-2bbab4aef6cd\") " pod="test-ns-feast/postgres-7c7656cc6d-lcwn9"
May 28 01:07:17.971946 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:17.971787    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr7nf\" (UniqueName: \"kubernetes.io/projected/f310bbd7-973a-4449-a5fc-2bbab4aef6cd-kube-api-access-qr7nf\") pod \"postgres-7c7656cc6d-lcwn9\" (UID: \"f310bbd7-973a-4449-a5fc-2bbab4aef6cd\") " pod="test-ns-feast/postgres-7c7656cc6d-lcwn9"
May 28 01:07:18.072564 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:18.072531    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nn2g\" (UniqueName: \"kubernetes.io/projected/fb3efc1c-c569-42f3-8c79-1d75fc5a6ede-kube-api-access-5nn2g\") pod \"redis-854544c579-xkw9w\" (UID: \"fb3efc1c-c569-42f3-8c79-1d75fc5a6ede\") " pod="test-ns-feast/redis-854544c579-xkw9w"
May 28 01:07:18.072660 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:18.072578    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/f310bbd7-973a-4449-a5fc-2bbab4aef6cd-postgresdata\") pod \"postgres-7c7656cc6d-lcwn9\" (UID: \"f310bbd7-973a-4449-a5fc-2bbab4aef6cd\") " pod="test-ns-feast/postgres-7c7656cc6d-lcwn9"
May 28 01:07:18.072660 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:18.072600    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qr7nf\" (UniqueName: \"kubernetes.io/projected/f310bbd7-973a-4449-a5fc-2bbab4aef6cd-kube-api-access-qr7nf\") pod \"postgres-7c7656cc6d-lcwn9\" (UID: \"f310bbd7-973a-4449-a5fc-2bbab4aef6cd\") " pod="test-ns-feast/postgres-7c7656cc6d-lcwn9"
May 28 01:07:18.072986 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:18.072967    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/f310bbd7-973a-4449-a5fc-2bbab4aef6cd-postgresdata\") pod \"postgres-7c7656cc6d-lcwn9\" (UID: \"f310bbd7-973a-4449-a5fc-2bbab4aef6cd\") " pod="test-ns-feast/postgres-7c7656cc6d-lcwn9"
May 28 01:07:18.079711 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:18.079684    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr7nf\" (UniqueName: \"kubernetes.io/projected/f310bbd7-973a-4449-a5fc-2bbab4aef6cd-kube-api-access-qr7nf\") pod \"postgres-7c7656cc6d-lcwn9\" (UID: \"f310bbd7-973a-4449-a5fc-2bbab4aef6cd\") " pod="test-ns-feast/postgres-7c7656cc6d-lcwn9"
May 28 01:07:18.173697 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:18.173663    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5nn2g\" (UniqueName: \"kubernetes.io/projected/fb3efc1c-c569-42f3-8c79-1d75fc5a6ede-kube-api-access-5nn2g\") pod \"redis-854544c579-xkw9w\" (UID: \"fb3efc1c-c569-42f3-8c79-1d75fc5a6ede\") " pod="test-ns-feast/redis-854544c579-xkw9w"
May 28 01:07:18.181269 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:18.181237    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nn2g\" (UniqueName: \"kubernetes.io/projected/fb3efc1c-c569-42f3-8c79-1d75fc5a6ede-kube-api-access-5nn2g\") pod \"redis-854544c579-xkw9w\" (UID: \"fb3efc1c-c569-42f3-8c79-1d75fc5a6ede\") " pod="test-ns-feast/redis-854544c579-xkw9w"
May 28 01:07:18.212224 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:18.212188    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-lcwn9"
May 28 01:07:18.239132 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:18.239099    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-854544c579-xkw9w"
May 28 01:07:18.350948 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:18.350918    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-lcwn9"]
May 28 01:07:18.354935 ip-10-0-131-73 kubenswrapper[2550]: W0528 01:07:18.354907    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf310bbd7_973a_4449_a5fc_2bbab4aef6cd.slice/crio-7c185e88f13b7cf9dda1733268c9edd26c1e2222c67c0baea4c8d44b13d30797 WatchSource:0}: Error finding container 7c185e88f13b7cf9dda1733268c9edd26c1e2222c67c0baea4c8d44b13d30797: Status 404 returned error can't find the container with id 7c185e88f13b7cf9dda1733268c9edd26c1e2222c67c0baea4c8d44b13d30797
May 28 01:07:18.356872 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:18.356851    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 01:07:18.374435 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:18.374385    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-854544c579-xkw9w"]
May 28 01:07:18.377478 ip-10-0-131-73 kubenswrapper[2550]: W0528 01:07:18.377452    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb3efc1c_c569_42f3_8c79_1d75fc5a6ede.slice/crio-157a3c9c3c7869dbc3ade5498265b54e36dc2cc0b0ee129882f9b27aab06e290 WatchSource:0}: Error finding container 157a3c9c3c7869dbc3ade5498265b54e36dc2cc0b0ee129882f9b27aab06e290: Status 404 returned error can't find the container with id 157a3c9c3c7869dbc3ade5498265b54e36dc2cc0b0ee129882f9b27aab06e290
May 28 01:07:18.486888 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:18.486792    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-854544c579-xkw9w" event={"ID":"fb3efc1c-c569-42f3-8c79-1d75fc5a6ede","Type":"ContainerStarted","Data":"157a3c9c3c7869dbc3ade5498265b54e36dc2cc0b0ee129882f9b27aab06e290"}
May 28 01:07:18.487802 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:18.487774    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-lcwn9" event={"ID":"f310bbd7-973a-4449-a5fc-2bbab4aef6cd","Type":"ContainerStarted","Data":"7c185e88f13b7cf9dda1733268c9edd26c1e2222c67c0baea4c8d44b13d30797"}
May 28 01:07:20.494725 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:20.494687    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-854544c579-xkw9w" event={"ID":"fb3efc1c-c569-42f3-8c79-1d75fc5a6ede","Type":"ContainerStarted","Data":"69d2f54e7b3dfa83ea37835921c1dc58e73a420fe0f355598acf8b38221e0f3f"}
May 28 01:07:20.509794 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:20.509710    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/redis-854544c579-xkw9w" podStartSLOduration=1.778518129 podStartE2EDuration="3.509690774s" podCreationTimestamp="2026-05-28 01:07:17 +0000 UTC" firstStartedPulling="2026-05-28 01:07:18.37910145 +0000 UTC m=+697.660058521" lastFinishedPulling="2026-05-28 01:07:20.110274085 +0000 UTC m=+699.391231166" observedRunningTime="2026-05-28 01:07:20.509325685 +0000 UTC m=+699.790282769" watchObservedRunningTime="2026-05-28 01:07:20.509690774 +0000 UTC m=+699.790647868"
May 28 01:07:25.509927 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:25.509888    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-lcwn9" event={"ID":"f310bbd7-973a-4449-a5fc-2bbab4aef6cd","Type":"ContainerStarted","Data":"78dba5f45143b21d2bad1551abdf9f725e403f24d74d8986997abca8ac379eb0"}
May 28 01:07:25.525428 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:25.525363    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/postgres-7c7656cc6d-lcwn9" podStartSLOduration=2.078842876 podStartE2EDuration="8.525346082s" podCreationTimestamp="2026-05-28 01:07:17 +0000 UTC" firstStartedPulling="2026-05-28 01:07:18.357017735 +0000 UTC m=+697.637974807" lastFinishedPulling="2026-05-28 01:07:24.803520943 +0000 UTC m=+704.084478013" observedRunningTime="2026-05-28 01:07:25.523241338 +0000 UTC m=+704.804198428" watchObservedRunningTime="2026-05-28 01:07:25.525346082 +0000 UTC m=+704.806303174"
May 28 01:07:28.593152 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:28.593122    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-864dc88f68-8splv"]
May 28 01:07:28.599824 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:28.599800    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:28.601737 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:28.601710    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\""
May 28 01:07:28.601737 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:28.601723    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-mkzsr\""
May 28 01:07:28.601737 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:28.601710    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\""
May 28 01:07:28.601981 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:28.601725    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\""
May 28 01:07:28.601981 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:28.601869    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\""
May 28 01:07:28.609861 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:28.609836    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-864dc88f68-8splv"]
May 28 01:07:28.764470 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:28.764404    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6qx9\" (UniqueName: \"kubernetes.io/projected/acf29cce-ee0d-42cd-9299-718c0c149b9b-kube-api-access-r6qx9\") pod \"feast-credit-scoring-864dc88f68-8splv\" (UID: \"acf29cce-ee0d-42cd-9299-718c0c149b9b\") " pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:28.764643 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:28.764520    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/acf29cce-ee0d-42cd-9299-718c0c149b9b-registry-tls\") pod \"feast-credit-scoring-864dc88f68-8splv\" (UID: \"acf29cce-ee0d-42cd-9299-718c0c149b9b\") " pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:28.764643 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:28.764581    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/acf29cce-ee0d-42cd-9299-718c0c149b9b-offline-tls\") pod \"feast-credit-scoring-864dc88f68-8splv\" (UID: \"acf29cce-ee0d-42cd-9299-718c0c149b9b\") " pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:28.764779 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:28.764677    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/acf29cce-ee0d-42cd-9299-718c0c149b9b-feast-data\") pod \"feast-credit-scoring-864dc88f68-8splv\" (UID: \"acf29cce-ee0d-42cd-9299-718c0c149b9b\") " pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:28.764779 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:28.764728    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/acf29cce-ee0d-42cd-9299-718c0c149b9b-online-tls\") pod \"feast-credit-scoring-864dc88f68-8splv\" (UID: \"acf29cce-ee0d-42cd-9299-718c0c149b9b\") " pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:28.865466 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:28.865345    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/acf29cce-ee0d-42cd-9299-718c0c149b9b-feast-data\") pod \"feast-credit-scoring-864dc88f68-8splv\" (UID: \"acf29cce-ee0d-42cd-9299-718c0c149b9b\") " pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:28.865466 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:28.865398    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/acf29cce-ee0d-42cd-9299-718c0c149b9b-online-tls\") pod \"feast-credit-scoring-864dc88f68-8splv\" (UID: \"acf29cce-ee0d-42cd-9299-718c0c149b9b\") " pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:28.865466 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:28.865456    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r6qx9\" (UniqueName: \"kubernetes.io/projected/acf29cce-ee0d-42cd-9299-718c0c149b9b-kube-api-access-r6qx9\") pod \"feast-credit-scoring-864dc88f68-8splv\" (UID: \"acf29cce-ee0d-42cd-9299-718c0c149b9b\") " pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:28.865751 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:28.865489    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/acf29cce-ee0d-42cd-9299-718c0c149b9b-registry-tls\") pod \"feast-credit-scoring-864dc88f68-8splv\" (UID: \"acf29cce-ee0d-42cd-9299-718c0c149b9b\") " pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:28.865751 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:28.865506    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/acf29cce-ee0d-42cd-9299-718c0c149b9b-offline-tls\") pod \"feast-credit-scoring-864dc88f68-8splv\" (UID: \"acf29cce-ee0d-42cd-9299-718c0c149b9b\") " pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:28.865845 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:28.865762    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/acf29cce-ee0d-42cd-9299-718c0c149b9b-feast-data\") pod \"feast-credit-scoring-864dc88f68-8splv\" (UID: \"acf29cce-ee0d-42cd-9299-718c0c149b9b\") " pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:28.867885 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:28.867855    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/acf29cce-ee0d-42cd-9299-718c0c149b9b-online-tls\") pod \"feast-credit-scoring-864dc88f68-8splv\" (UID: \"acf29cce-ee0d-42cd-9299-718c0c149b9b\") " pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:28.868023 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:28.867926    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/acf29cce-ee0d-42cd-9299-718c0c149b9b-offline-tls\") pod \"feast-credit-scoring-864dc88f68-8splv\" (UID: \"acf29cce-ee0d-42cd-9299-718c0c149b9b\") " pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:28.868023 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:28.867937    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/acf29cce-ee0d-42cd-9299-718c0c149b9b-registry-tls\") pod \"feast-credit-scoring-864dc88f68-8splv\" (UID: \"acf29cce-ee0d-42cd-9299-718c0c149b9b\") " pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:28.873950 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:28.873933    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6qx9\" (UniqueName: \"kubernetes.io/projected/acf29cce-ee0d-42cd-9299-718c0c149b9b-kube-api-access-r6qx9\") pod \"feast-credit-scoring-864dc88f68-8splv\" (UID: \"acf29cce-ee0d-42cd-9299-718c0c149b9b\") " pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:28.909732 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:28.909695    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:29.043369 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:29.043315    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-864dc88f68-8splv"]
May 28 01:07:29.048199 ip-10-0-131-73 kubenswrapper[2550]: W0528 01:07:29.048157    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacf29cce_ee0d_42cd_9299_718c0c149b9b.slice/crio-b558413ea78ff105940a474d9cb4c4bafa15a073f57c2a106ea6ef10553a6018 WatchSource:0}: Error finding container b558413ea78ff105940a474d9cb4c4bafa15a073f57c2a106ea6ef10553a6018: Status 404 returned error can't find the container with id b558413ea78ff105940a474d9cb4c4bafa15a073f57c2a106ea6ef10553a6018
May 28 01:07:29.521961 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:29.521915    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv" event={"ID":"acf29cce-ee0d-42cd-9299-718c0c149b9b","Type":"ContainerStarted","Data":"72c6687c2e0da3b8c9c0abb8a8ae4839e353842304363c52ba7c60859cabe2a5"}
May 28 01:07:29.522131 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:29.521980    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv" event={"ID":"acf29cce-ee0d-42cd-9299-718c0c149b9b","Type":"ContainerStarted","Data":"b558413ea78ff105940a474d9cb4c4bafa15a073f57c2a106ea6ef10553a6018"}
May 28 01:07:30.525713 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:30.525680    2550 generic.go:358] "Generic (PLEG): container finished" podID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerID="72c6687c2e0da3b8c9c0abb8a8ae4839e353842304363c52ba7c60859cabe2a5" exitCode=0
May 28 01:07:30.526080 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:30.525758    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv" event={"ID":"acf29cce-ee0d-42cd-9299-718c0c149b9b","Type":"ContainerDied","Data":"72c6687c2e0da3b8c9c0abb8a8ae4839e353842304363c52ba7c60859cabe2a5"}
May 28 01:07:31.530353 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:31.530314    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv" event={"ID":"acf29cce-ee0d-42cd-9299-718c0c149b9b","Type":"ContainerStarted","Data":"38cccf498a885906ba18e0b1435784ae2f7ae0fb42f061c687aaf337cc825923"}
May 28 01:07:37.549121 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:37.549088    2550 generic.go:358] "Generic (PLEG): container finished" podID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerID="38cccf498a885906ba18e0b1435784ae2f7ae0fb42f061c687aaf337cc825923" exitCode=0
May 28 01:07:37.549540 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:37.549162    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv" event={"ID":"acf29cce-ee0d-42cd-9299-718c0c149b9b","Type":"ContainerDied","Data":"38cccf498a885906ba18e0b1435784ae2f7ae0fb42f061c687aaf337cc825923"}
May 28 01:07:38.556162 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:38.556124    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv" event={"ID":"acf29cce-ee0d-42cd-9299-718c0c149b9b","Type":"ContainerStarted","Data":"cbb668f500e458dd1367a1ad621d600bea0f66979ee9d70548e54f94b1c68f1a"}
May 28 01:07:38.556162 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:38.556170    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv" event={"ID":"acf29cce-ee0d-42cd-9299-718c0c149b9b","Type":"ContainerStarted","Data":"25f3acc1be2a31c558221330e455dc1a0d25891d111aabc5edacc110d572f873"}
May 28 01:07:38.556733 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:38.556181    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv" event={"ID":"acf29cce-ee0d-42cd-9299-718c0c149b9b","Type":"ContainerStarted","Data":"7ea59b093154db1157e075649297b9f8ab0f31cf1bc0326861e9344936a0d8b5"}
May 28 01:07:38.579618 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:38.579556    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv" podStartSLOduration=10.579535094 podStartE2EDuration="10.579535094s" podCreationTimestamp="2026-05-28 01:07:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 01:07:38.577045836 +0000 UTC m=+717.858002928" watchObservedRunningTime="2026-05-28 01:07:38.579535094 +0000 UTC m=+717.860492186"
May 28 01:07:38.910776 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:38.910681    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:38.910776 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:38.910749    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:38.910776 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:38.910767    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:40.910076 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:40.910031    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:40.910076 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:40.910084    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:40.910666 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:40.910099    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:40.911646 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:40.911606    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv" podUID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.24:6571: connect: connection refused"
May 28 01:07:40.911764 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:40.911648    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv" podUID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused"
May 28 01:07:40.911764 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:40.911625    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv" podUID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.24:8816: connect: connection refused"
May 28 01:07:43.910341 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:43.910283    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv" podUID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.24:6571: connect: connection refused"
May 28 01:07:43.911047 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:43.911016    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:43.911928 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:43.911905    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:43.915809 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:43.915781    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:44.580533 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:44.580500    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:46.910986 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:46.910939    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv" podUID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.24:6571: connect: connection refused"
May 28 01:07:49.910723 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:49.910688    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:07:49.911381 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:07:49.911359    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:08:51.416346 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.416308    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-864dc88f68-8splv"]
May 28 01:08:51.416900 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.416708    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv" podUID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerName="registry" containerID="cri-o://7ea59b093154db1157e075649297b9f8ab0f31cf1bc0326861e9344936a0d8b5" gracePeriod=30
May 28 01:08:51.416900 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.416757    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv" podUID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerName="offline" containerID="cri-o://cbb668f500e458dd1367a1ad621d600bea0f66979ee9d70548e54f94b1c68f1a" gracePeriod=30
May 28 01:08:51.416900 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.416863    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv" podUID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerName="online" containerID="cri-o://25f3acc1be2a31c558221330e455dc1a0d25891d111aabc5edacc110d572f873" gracePeriod=30
May 28 01:08:51.434526 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.434485    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-lcwn9"]
May 28 01:08:51.434759 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.434718    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/postgres-7c7656cc6d-lcwn9" podUID="f310bbd7-973a-4449-a5fc-2bbab4aef6cd" containerName="postgres" containerID="cri-o://78dba5f45143b21d2bad1551abdf9f725e403f24d74d8986997abca8ac379eb0" gracePeriod=30
May 28 01:08:51.438822 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.438796    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-854544c579-xkw9w"]
May 28 01:08:51.439004 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.438984    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/redis-854544c579-xkw9w" podUID="fb3efc1c-c569-42f3-8c79-1d75fc5a6ede" containerName="redis" containerID="cri-o://69d2f54e7b3dfa83ea37835921c1dc58e73a420fe0f355598acf8b38221e0f3f" gracePeriod=30
May 28 01:08:51.702692 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.702669    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-lcwn9"
May 28 01:08:51.705619 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.705600    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-854544c579-xkw9w"
May 28 01:08:51.762587 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.762555    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/f310bbd7-973a-4449-a5fc-2bbab4aef6cd-postgresdata\") pod \"f310bbd7-973a-4449-a5fc-2bbab4aef6cd\" (UID: \"f310bbd7-973a-4449-a5fc-2bbab4aef6cd\") "
May 28 01:08:51.762771 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.762638    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qr7nf\" (UniqueName: \"kubernetes.io/projected/f310bbd7-973a-4449-a5fc-2bbab4aef6cd-kube-api-access-qr7nf\") pod \"f310bbd7-973a-4449-a5fc-2bbab4aef6cd\" (UID: \"f310bbd7-973a-4449-a5fc-2bbab4aef6cd\") "
May 28 01:08:51.762771 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.762669    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5nn2g\" (UniqueName: \"kubernetes.io/projected/fb3efc1c-c569-42f3-8c79-1d75fc5a6ede-kube-api-access-5nn2g\") pod \"fb3efc1c-c569-42f3-8c79-1d75fc5a6ede\" (UID: \"fb3efc1c-c569-42f3-8c79-1d75fc5a6ede\") "
May 28 01:08:51.765181 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.765148    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f310bbd7-973a-4449-a5fc-2bbab4aef6cd-kube-api-access-qr7nf" (OuterVolumeSpecName: "kube-api-access-qr7nf") pod "f310bbd7-973a-4449-a5fc-2bbab4aef6cd" (UID: "f310bbd7-973a-4449-a5fc-2bbab4aef6cd"). InnerVolumeSpecName "kube-api-access-qr7nf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 01:08:51.765345 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.765215    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb3efc1c-c569-42f3-8c79-1d75fc5a6ede-kube-api-access-5nn2g" (OuterVolumeSpecName: "kube-api-access-5nn2g") pod "fb3efc1c-c569-42f3-8c79-1d75fc5a6ede" (UID: "fb3efc1c-c569-42f3-8c79-1d75fc5a6ede"). InnerVolumeSpecName "kube-api-access-5nn2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 01:08:51.773761 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.773731    2550 generic.go:358] "Generic (PLEG): container finished" podID="fb3efc1c-c569-42f3-8c79-1d75fc5a6ede" containerID="69d2f54e7b3dfa83ea37835921c1dc58e73a420fe0f355598acf8b38221e0f3f" exitCode=0
May 28 01:08:51.773904 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.773792    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-854544c579-xkw9w"
May 28 01:08:51.773904 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.773817    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-854544c579-xkw9w" event={"ID":"fb3efc1c-c569-42f3-8c79-1d75fc5a6ede","Type":"ContainerDied","Data":"69d2f54e7b3dfa83ea37835921c1dc58e73a420fe0f355598acf8b38221e0f3f"}
May 28 01:08:51.773904 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.773854    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-854544c579-xkw9w" event={"ID":"fb3efc1c-c569-42f3-8c79-1d75fc5a6ede","Type":"ContainerDied","Data":"157a3c9c3c7869dbc3ade5498265b54e36dc2cc0b0ee129882f9b27aab06e290"}
May 28 01:08:51.773904 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.773875    2550 scope.go:117] "RemoveContainer" containerID="69d2f54e7b3dfa83ea37835921c1dc58e73a420fe0f355598acf8b38221e0f3f"
May 28 01:08:51.775063 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.775042    2550 generic.go:358] "Generic (PLEG): container finished" podID="f310bbd7-973a-4449-a5fc-2bbab4aef6cd" containerID="78dba5f45143b21d2bad1551abdf9f725e403f24d74d8986997abca8ac379eb0" exitCode=0
May 28 01:08:51.775165 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.775095    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-lcwn9"
May 28 01:08:51.775165 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.775115    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-lcwn9" event={"ID":"f310bbd7-973a-4449-a5fc-2bbab4aef6cd","Type":"ContainerDied","Data":"78dba5f45143b21d2bad1551abdf9f725e403f24d74d8986997abca8ac379eb0"}
May 28 01:08:51.775165 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.775141    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-lcwn9" event={"ID":"f310bbd7-973a-4449-a5fc-2bbab4aef6cd","Type":"ContainerDied","Data":"7c185e88f13b7cf9dda1733268c9edd26c1e2222c67c0baea4c8d44b13d30797"}
May 28 01:08:51.782119 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.782094    2550 scope.go:117] "RemoveContainer" containerID="69d2f54e7b3dfa83ea37835921c1dc58e73a420fe0f355598acf8b38221e0f3f"
May 28 01:08:51.782501 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:08:51.782440    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69d2f54e7b3dfa83ea37835921c1dc58e73a420fe0f355598acf8b38221e0f3f\": container with ID starting with 69d2f54e7b3dfa83ea37835921c1dc58e73a420fe0f355598acf8b38221e0f3f not found: ID does not exist" containerID="69d2f54e7b3dfa83ea37835921c1dc58e73a420fe0f355598acf8b38221e0f3f"
May 28 01:08:51.782501 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.782477    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69d2f54e7b3dfa83ea37835921c1dc58e73a420fe0f355598acf8b38221e0f3f"} err="failed to get container status \"69d2f54e7b3dfa83ea37835921c1dc58e73a420fe0f355598acf8b38221e0f3f\": rpc error: code = NotFound desc = could not find container \"69d2f54e7b3dfa83ea37835921c1dc58e73a420fe0f355598acf8b38221e0f3f\": container with ID starting with 69d2f54e7b3dfa83ea37835921c1dc58e73a420fe0f355598acf8b38221e0f3f not found: ID does not exist"
May 28 01:08:51.782501 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.782501    2550 scope.go:117] "RemoveContainer" containerID="78dba5f45143b21d2bad1551abdf9f725e403f24d74d8986997abca8ac379eb0"
May 28 01:08:51.790923 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.790900    2550 scope.go:117] "RemoveContainer" containerID="78dba5f45143b21d2bad1551abdf9f725e403f24d74d8986997abca8ac379eb0"
May 28 01:08:51.791187 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:08:51.791168    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78dba5f45143b21d2bad1551abdf9f725e403f24d74d8986997abca8ac379eb0\": container with ID starting with 78dba5f45143b21d2bad1551abdf9f725e403f24d74d8986997abca8ac379eb0 not found: ID does not exist" containerID="78dba5f45143b21d2bad1551abdf9f725e403f24d74d8986997abca8ac379eb0"
May 28 01:08:51.791263 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.791197    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78dba5f45143b21d2bad1551abdf9f725e403f24d74d8986997abca8ac379eb0"} err="failed to get container status \"78dba5f45143b21d2bad1551abdf9f725e403f24d74d8986997abca8ac379eb0\": rpc error: code = NotFound desc = could not find container \"78dba5f45143b21d2bad1551abdf9f725e403f24d74d8986997abca8ac379eb0\": container with ID starting with 78dba5f45143b21d2bad1551abdf9f725e403f24d74d8986997abca8ac379eb0 not found: ID does not exist"
May 28 01:08:51.793684 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.793664    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-854544c579-xkw9w"]
May 28 01:08:51.797144 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.797118    2550 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-854544c579-xkw9w"]
May 28 01:08:51.797254 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.797232    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f310bbd7-973a-4449-a5fc-2bbab4aef6cd-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "f310bbd7-973a-4449-a5fc-2bbab4aef6cd" (UID: "f310bbd7-973a-4449-a5fc-2bbab4aef6cd"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 01:08:51.863655 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.863619    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qr7nf\" (UniqueName: \"kubernetes.io/projected/f310bbd7-973a-4449-a5fc-2bbab4aef6cd-kube-api-access-qr7nf\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:08:51.863655 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.863647    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5nn2g\" (UniqueName: \"kubernetes.io/projected/fb3efc1c-c569-42f3-8c79-1d75fc5a6ede-kube-api-access-5nn2g\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:08:51.863655 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:51.863659    2550 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/f310bbd7-973a-4449-a5fc-2bbab4aef6cd-postgresdata\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:08:52.099210 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:52.099177    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-lcwn9"]
May 28 01:08:52.102453 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:52.102409    2550 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-lcwn9"]
May 28 01:08:52.780226 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:52.780123    2550 generic.go:358] "Generic (PLEG): container finished" podID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerID="25f3acc1be2a31c558221330e455dc1a0d25891d111aabc5edacc110d572f873" exitCode=0
May 28 01:08:52.780226 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:52.780204    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv" event={"ID":"acf29cce-ee0d-42cd-9299-718c0c149b9b","Type":"ContainerDied","Data":"25f3acc1be2a31c558221330e455dc1a0d25891d111aabc5edacc110d572f873"}
May 28 01:08:53.324140 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:53.324102    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f310bbd7-973a-4449-a5fc-2bbab4aef6cd" path="/var/lib/kubelet/pods/f310bbd7-973a-4449-a5fc-2bbab4aef6cd/volumes"
May 28 01:08:53.324440 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:53.324408    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb3efc1c-c569-42f3-8c79-1d75fc5a6ede" path="/var/lib/kubelet/pods/fb3efc1c-c569-42f3-8c79-1d75fc5a6ede/volumes"
May 28 01:08:54.578478 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:08:54.578408    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv" podUID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused"
May 28 01:09:04.577591 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:04.577544    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv" podUID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused"
May 28 01:09:14.578166 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:14.578118    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv" podUID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused"
May 28 01:09:14.578581 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:14.578261    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:09:21.861166 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:21.861087    2550 generic.go:358] "Generic (PLEG): container finished" podID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerID="cbb668f500e458dd1367a1ad621d600bea0f66979ee9d70548e54f94b1c68f1a" exitCode=137
May 28 01:09:21.861166 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:21.861115    2550 generic.go:358] "Generic (PLEG): container finished" podID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerID="7ea59b093154db1157e075649297b9f8ab0f31cf1bc0326861e9344936a0d8b5" exitCode=137
May 28 01:09:21.861166 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:21.861156    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv" event={"ID":"acf29cce-ee0d-42cd-9299-718c0c149b9b","Type":"ContainerDied","Data":"cbb668f500e458dd1367a1ad621d600bea0f66979ee9d70548e54f94b1c68f1a"}
May 28 01:09:21.861594 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:21.861189    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv" event={"ID":"acf29cce-ee0d-42cd-9299-718c0c149b9b","Type":"ContainerDied","Data":"7ea59b093154db1157e075649297b9f8ab0f31cf1bc0326861e9344936a0d8b5"}
May 28 01:09:22.062471 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:22.062448    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:09:22.213318 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:22.213283    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/acf29cce-ee0d-42cd-9299-718c0c149b9b-feast-data\") pod \"acf29cce-ee0d-42cd-9299-718c0c149b9b\" (UID: \"acf29cce-ee0d-42cd-9299-718c0c149b9b\") "
May 28 01:09:22.213513 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:22.213327    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6qx9\" (UniqueName: \"kubernetes.io/projected/acf29cce-ee0d-42cd-9299-718c0c149b9b-kube-api-access-r6qx9\") pod \"acf29cce-ee0d-42cd-9299-718c0c149b9b\" (UID: \"acf29cce-ee0d-42cd-9299-718c0c149b9b\") "
May 28 01:09:22.213513 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:22.213357    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/acf29cce-ee0d-42cd-9299-718c0c149b9b-registry-tls\") pod \"acf29cce-ee0d-42cd-9299-718c0c149b9b\" (UID: \"acf29cce-ee0d-42cd-9299-718c0c149b9b\") "
May 28 01:09:22.213513 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:22.213377    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/acf29cce-ee0d-42cd-9299-718c0c149b9b-online-tls\") pod \"acf29cce-ee0d-42cd-9299-718c0c149b9b\" (UID: \"acf29cce-ee0d-42cd-9299-718c0c149b9b\") "
May 28 01:09:22.213658 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:22.213515    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/acf29cce-ee0d-42cd-9299-718c0c149b9b-offline-tls\") pod \"acf29cce-ee0d-42cd-9299-718c0c149b9b\" (UID: \"acf29cce-ee0d-42cd-9299-718c0c149b9b\") "
May 28 01:09:22.215565 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:22.215537    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acf29cce-ee0d-42cd-9299-718c0c149b9b-online-tls" (OuterVolumeSpecName: "online-tls") pod "acf29cce-ee0d-42cd-9299-718c0c149b9b" (UID: "acf29cce-ee0d-42cd-9299-718c0c149b9b"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:09:22.215680 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:22.215559    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acf29cce-ee0d-42cd-9299-718c0c149b9b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "acf29cce-ee0d-42cd-9299-718c0c149b9b" (UID: "acf29cce-ee0d-42cd-9299-718c0c149b9b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:09:22.215680 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:22.215537    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acf29cce-ee0d-42cd-9299-718c0c149b9b-kube-api-access-r6qx9" (OuterVolumeSpecName: "kube-api-access-r6qx9") pod "acf29cce-ee0d-42cd-9299-718c0c149b9b" (UID: "acf29cce-ee0d-42cd-9299-718c0c149b9b"). InnerVolumeSpecName "kube-api-access-r6qx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 01:09:22.216136 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:22.216117    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acf29cce-ee0d-42cd-9299-718c0c149b9b-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "acf29cce-ee0d-42cd-9299-718c0c149b9b" (UID: "acf29cce-ee0d-42cd-9299-718c0c149b9b"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:09:22.222075 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:22.222049    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acf29cce-ee0d-42cd-9299-718c0c149b9b-feast-data" (OuterVolumeSpecName: "feast-data") pod "acf29cce-ee0d-42cd-9299-718c0c149b9b" (UID: "acf29cce-ee0d-42cd-9299-718c0c149b9b"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 01:09:22.314804 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:22.314769    2550 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/acf29cce-ee0d-42cd-9299-718c0c149b9b-feast-data\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:09:22.314804 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:22.314799    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6qx9\" (UniqueName: \"kubernetes.io/projected/acf29cce-ee0d-42cd-9299-718c0c149b9b-kube-api-access-r6qx9\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:09:22.314804 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:22.314811    2550 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/acf29cce-ee0d-42cd-9299-718c0c149b9b-registry-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:09:22.315006 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:22.314820    2550 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/acf29cce-ee0d-42cd-9299-718c0c149b9b-online-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:09:22.315006 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:22.314829    2550 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/acf29cce-ee0d-42cd-9299-718c0c149b9b-offline-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:09:22.865849 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:22.865810    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv" event={"ID":"acf29cce-ee0d-42cd-9299-718c0c149b9b","Type":"ContainerDied","Data":"b558413ea78ff105940a474d9cb4c4bafa15a073f57c2a106ea6ef10553a6018"}
May 28 01:09:22.866271 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:22.865862    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-864dc88f68-8splv"
May 28 01:09:22.866271 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:22.865866    2550 scope.go:117] "RemoveContainer" containerID="cbb668f500e458dd1367a1ad621d600bea0f66979ee9d70548e54f94b1c68f1a"
May 28 01:09:22.874204 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:22.874185    2550 scope.go:117] "RemoveContainer" containerID="25f3acc1be2a31c558221330e455dc1a0d25891d111aabc5edacc110d572f873"
May 28 01:09:22.881227 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:22.881210    2550 scope.go:117] "RemoveContainer" containerID="7ea59b093154db1157e075649297b9f8ab0f31cf1bc0326861e9344936a0d8b5"
May 28 01:09:22.887079 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:22.887047    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-864dc88f68-8splv"]
May 28 01:09:22.888127 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:22.888110    2550 scope.go:117] "RemoveContainer" containerID="38cccf498a885906ba18e0b1435784ae2f7ae0fb42f061c687aaf337cc825923"
May 28 01:09:22.890719 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:22.890696    2550 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-864dc88f68-8splv"]
May 28 01:09:22.894999 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:22.894981    2550 scope.go:117] "RemoveContainer" containerID="72c6687c2e0da3b8c9c0abb8a8ae4839e353842304363c52ba7c60859cabe2a5"
May 28 01:09:23.323839 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:09:23.323795    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acf29cce-ee0d-42cd-9299-718c0c149b9b" path="/var/lib/kubelet/pods/acf29cce-ee0d-42cd-9299-718c0c149b9b/volumes"
May 28 01:10:41.267589 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:10:41.267514    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9z98m_6671c1c1-b674-4a1d-9011-1ca22724b234/ovn-acl-logging/0.log"
May 28 01:10:41.273492 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:10:41.273466    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9z98m_6671c1c1-b674-4a1d-9011-1ca22724b234/ovn-acl-logging/0.log"
May 28 01:11:11.347503 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.347468    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-854544c579-6ssbx"]
May 28 01:11:11.347879 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.347724    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb3efc1c-c569-42f3-8c79-1d75fc5a6ede" containerName="redis"
May 28 01:11:11.347879 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.347737    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb3efc1c-c569-42f3-8c79-1d75fc5a6ede" containerName="redis"
May 28 01:11:11.347879 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.347748    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerName="online"
May 28 01:11:11.347879 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.347753    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerName="online"
May 28 01:11:11.347879 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.347760    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f310bbd7-973a-4449-a5fc-2bbab4aef6cd" containerName="postgres"
May 28 01:11:11.347879 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.347766    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="f310bbd7-973a-4449-a5fc-2bbab4aef6cd" containerName="postgres"
May 28 01:11:11.347879 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.347777    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerName="registry"
May 28 01:11:11.347879 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.347782    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerName="registry"
May 28 01:11:11.347879 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.347790    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerName="feast-init"
May 28 01:11:11.347879 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.347799    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerName="feast-init"
May 28 01:11:11.347879 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.347810    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerName="feast-apply"
May 28 01:11:11.347879 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.347815    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerName="feast-apply"
May 28 01:11:11.347879 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.347824    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerName="offline"
May 28 01:11:11.347879 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.347830    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerName="offline"
May 28 01:11:11.347879 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.347872    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerName="online"
May 28 01:11:11.347879 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.347883    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="fb3efc1c-c569-42f3-8c79-1d75fc5a6ede" containerName="redis"
May 28 01:11:11.348386 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.347890    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="f310bbd7-973a-4449-a5fc-2bbab4aef6cd" containerName="postgres"
May 28 01:11:11.348386 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.347896    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerName="offline"
May 28 01:11:11.348386 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.347901    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="acf29cce-ee0d-42cd-9299-718c0c149b9b" containerName="registry"
May 28 01:11:11.350683 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.350663    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-6ssbx"
May 28 01:11:11.352663 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.352639    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\""
May 28 01:11:11.353262 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.353239    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\""
May 28 01:11:11.353466 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.353241    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-6m9lj\""
May 28 01:11:11.365025 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.364998    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-854544c579-6ssbx"]
May 28 01:11:11.456478 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.456442    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zw9jf\" (UniqueName: \"kubernetes.io/projected/664b6570-2c2e-4260-b2a9-5425eeddbec1-kube-api-access-zw9jf\") pod \"redis-854544c579-6ssbx\" (UID: \"664b6570-2c2e-4260-b2a9-5425eeddbec1\") " pod="test-ns-feast-rest/redis-854544c579-6ssbx"
May 28 01:11:11.557377 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.557330    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zw9jf\" (UniqueName: \"kubernetes.io/projected/664b6570-2c2e-4260-b2a9-5425eeddbec1-kube-api-access-zw9jf\") pod \"redis-854544c579-6ssbx\" (UID: \"664b6570-2c2e-4260-b2a9-5425eeddbec1\") " pod="test-ns-feast-rest/redis-854544c579-6ssbx"
May 28 01:11:11.567515 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.567493    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zw9jf\" (UniqueName: \"kubernetes.io/projected/664b6570-2c2e-4260-b2a9-5425eeddbec1-kube-api-access-zw9jf\") pod \"redis-854544c579-6ssbx\" (UID: \"664b6570-2c2e-4260-b2a9-5425eeddbec1\") " pod="test-ns-feast-rest/redis-854544c579-6ssbx"
May 28 01:11:11.671785 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.671741    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-6ssbx"
May 28 01:11:11.797694 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:11.797664    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-854544c579-6ssbx"]
May 28 01:11:11.800436 ip-10-0-131-73 kubenswrapper[2550]: W0528 01:11:11.800384    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod664b6570_2c2e_4260_b2a9_5425eeddbec1.slice/crio-b771f6ac6027a1eeba46533f5e81472fba05e4c14d6a82f6537dd35f9d996690 WatchSource:0}: Error finding container b771f6ac6027a1eeba46533f5e81472fba05e4c14d6a82f6537dd35f9d996690: Status 404 returned error can't find the container with id b771f6ac6027a1eeba46533f5e81472fba05e4c14d6a82f6537dd35f9d996690
May 28 01:11:12.168138 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:12.168105    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-6ssbx" event={"ID":"664b6570-2c2e-4260-b2a9-5425eeddbec1","Type":"ContainerStarted","Data":"17fd31d37bd954aeb2e98cb7590fe962480e3841f1fd2aa172b037201eae17d1"}
May 28 01:11:12.168138 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:12.168139    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-6ssbx" event={"ID":"664b6570-2c2e-4260-b2a9-5425eeddbec1","Type":"ContainerStarted","Data":"b771f6ac6027a1eeba46533f5e81472fba05e4c14d6a82f6537dd35f9d996690"}
May 28 01:11:12.185046 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:12.184994    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/redis-854544c579-6ssbx" podStartSLOduration=1.184975836 podStartE2EDuration="1.184975836s" podCreationTimestamp="2026-05-28 01:11:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 01:11:12.184573653 +0000 UTC m=+931.465530745" watchObservedRunningTime="2026-05-28 01:11:12.184975836 +0000 UTC m=+931.465932929"
May 28 01:11:16.781557 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:16.781521    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-bgkbm"]
May 28 01:11:16.785050 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:16.785026    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-bgkbm"
May 28 01:11:16.787175 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:16.787151    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\""
May 28 01:11:16.790056 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:16.790031    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-bgkbm"]
May 28 01:11:16.898919 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:16.898878    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4hqx\" (UniqueName: \"kubernetes.io/projected/186635cb-ee5f-4051-9b3f-4d2228e1e048-kube-api-access-k4hqx\") pod \"postgres-7c7656cc6d-bgkbm\" (UID: \"186635cb-ee5f-4051-9b3f-4d2228e1e048\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-bgkbm"
May 28 01:11:16.899103 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:16.898927    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/186635cb-ee5f-4051-9b3f-4d2228e1e048-postgresdata\") pod \"postgres-7c7656cc6d-bgkbm\" (UID: \"186635cb-ee5f-4051-9b3f-4d2228e1e048\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-bgkbm"
May 28 01:11:16.999714 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:16.999683    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/186635cb-ee5f-4051-9b3f-4d2228e1e048-postgresdata\") pod \"postgres-7c7656cc6d-bgkbm\" (UID: \"186635cb-ee5f-4051-9b3f-4d2228e1e048\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-bgkbm"
May 28 01:11:16.999822 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:16.999780    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k4hqx\" (UniqueName: \"kubernetes.io/projected/186635cb-ee5f-4051-9b3f-4d2228e1e048-kube-api-access-k4hqx\") pod \"postgres-7c7656cc6d-bgkbm\" (UID: \"186635cb-ee5f-4051-9b3f-4d2228e1e048\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-bgkbm"
May 28 01:11:17.000142 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:17.000121    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/186635cb-ee5f-4051-9b3f-4d2228e1e048-postgresdata\") pod \"postgres-7c7656cc6d-bgkbm\" (UID: \"186635cb-ee5f-4051-9b3f-4d2228e1e048\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-bgkbm"
May 28 01:11:17.008309 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:17.008278    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4hqx\" (UniqueName: \"kubernetes.io/projected/186635cb-ee5f-4051-9b3f-4d2228e1e048-kube-api-access-k4hqx\") pod \"postgres-7c7656cc6d-bgkbm\" (UID: \"186635cb-ee5f-4051-9b3f-4d2228e1e048\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-bgkbm"
May 28 01:11:17.095376 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:17.095285    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-bgkbm"
May 28 01:11:17.219998 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:17.219963    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-bgkbm"]
May 28 01:11:17.223640 ip-10-0-131-73 kubenswrapper[2550]: W0528 01:11:17.223611    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod186635cb_ee5f_4051_9b3f_4d2228e1e048.slice/crio-4378ed20d9238924bbec88cbc1a9f0843987b1bfbf00ba83131b41976dfa5a97 WatchSource:0}: Error finding container 4378ed20d9238924bbec88cbc1a9f0843987b1bfbf00ba83131b41976dfa5a97: Status 404 returned error can't find the container with id 4378ed20d9238924bbec88cbc1a9f0843987b1bfbf00ba83131b41976dfa5a97
May 28 01:11:18.185907 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:18.185863    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-bgkbm" event={"ID":"186635cb-ee5f-4051-9b3f-4d2228e1e048","Type":"ContainerStarted","Data":"718008af9c82375013882ddf420f039e147f6ae80b45e29bcadeefc2044c2675"}
May 28 01:11:18.185907 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:18.185903    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-bgkbm" event={"ID":"186635cb-ee5f-4051-9b3f-4d2228e1e048","Type":"ContainerStarted","Data":"4378ed20d9238924bbec88cbc1a9f0843987b1bfbf00ba83131b41976dfa5a97"}
May 28 01:11:18.201701 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:18.201655    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-7c7656cc6d-bgkbm" podStartSLOduration=2.201640017 podStartE2EDuration="2.201640017s" podCreationTimestamp="2026-05-28 01:11:16 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 01:11:18.20061185 +0000 UTC m=+937.481568942" watchObservedRunningTime="2026-05-28 01:11:18.201640017 +0000 UTC m=+937.482597110"
May 28 01:11:22.441308 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:22.441272    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"]
May 28 01:11:22.443987 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:22.443962    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"
May 28 01:11:22.446526 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:22.446499    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-x62pp\""
May 28 01:11:22.446792 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:22.446773    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\""
May 28 01:11:22.446885 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:22.446813    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\""
May 28 01:11:22.446885 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:22.446814    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\""
May 28 01:11:22.456487 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:22.456466    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"]
May 28 01:11:22.548664 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:22.548630    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/72009548-a26c-414b-a747-fdb65a2707f8-online-tls\") pod \"feast-credit-scoring-fd659f88c-wn5hv\" (UID: \"72009548-a26c-414b-a747-fdb65a2707f8\") " pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"
May 28 01:11:22.548824 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:22.548674    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjgg2\" (UniqueName: \"kubernetes.io/projected/72009548-a26c-414b-a747-fdb65a2707f8-kube-api-access-sjgg2\") pod \"feast-credit-scoring-fd659f88c-wn5hv\" (UID: \"72009548-a26c-414b-a747-fdb65a2707f8\") " pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"
May 28 01:11:22.548824 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:22.548788    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/72009548-a26c-414b-a747-fdb65a2707f8-registry-tls\") pod \"feast-credit-scoring-fd659f88c-wn5hv\" (UID: \"72009548-a26c-414b-a747-fdb65a2707f8\") " pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"
May 28 01:11:22.548824 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:22.548813    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/72009548-a26c-414b-a747-fdb65a2707f8-feast-data\") pod \"feast-credit-scoring-fd659f88c-wn5hv\" (UID: \"72009548-a26c-414b-a747-fdb65a2707f8\") " pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"
May 28 01:11:22.649819 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:22.649783    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/72009548-a26c-414b-a747-fdb65a2707f8-registry-tls\") pod \"feast-credit-scoring-fd659f88c-wn5hv\" (UID: \"72009548-a26c-414b-a747-fdb65a2707f8\") " pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"
May 28 01:11:22.649819 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:22.649822    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/72009548-a26c-414b-a747-fdb65a2707f8-feast-data\") pod \"feast-credit-scoring-fd659f88c-wn5hv\" (UID: \"72009548-a26c-414b-a747-fdb65a2707f8\") " pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"
May 28 01:11:22.650063 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:22.649952    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/72009548-a26c-414b-a747-fdb65a2707f8-online-tls\") pod \"feast-credit-scoring-fd659f88c-wn5hv\" (UID: \"72009548-a26c-414b-a747-fdb65a2707f8\") " pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"
May 28 01:11:22.650063 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:22.650001    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sjgg2\" (UniqueName: \"kubernetes.io/projected/72009548-a26c-414b-a747-fdb65a2707f8-kube-api-access-sjgg2\") pod \"feast-credit-scoring-fd659f88c-wn5hv\" (UID: \"72009548-a26c-414b-a747-fdb65a2707f8\") " pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"
May 28 01:11:22.650173 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:11:22.650069    2550 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-credit-scoring-online-tls: secret "feast-credit-scoring-online-tls" not found
May 28 01:11:22.650173 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:11:22.650147    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72009548-a26c-414b-a747-fdb65a2707f8-online-tls podName:72009548-a26c-414b-a747-fdb65a2707f8 nodeName:}" failed. No retries permitted until 2026-05-28 01:11:23.150128054 +0000 UTC m=+942.431085144 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/72009548-a26c-414b-a747-fdb65a2707f8-online-tls") pod "feast-credit-scoring-fd659f88c-wn5hv" (UID: "72009548-a26c-414b-a747-fdb65a2707f8") : secret "feast-credit-scoring-online-tls" not found
May 28 01:11:22.650279 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:22.650203    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/72009548-a26c-414b-a747-fdb65a2707f8-feast-data\") pod \"feast-credit-scoring-fd659f88c-wn5hv\" (UID: \"72009548-a26c-414b-a747-fdb65a2707f8\") " pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"
May 28 01:11:22.652213 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:22.652194    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/72009548-a26c-414b-a747-fdb65a2707f8-registry-tls\") pod \"feast-credit-scoring-fd659f88c-wn5hv\" (UID: \"72009548-a26c-414b-a747-fdb65a2707f8\") " pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"
May 28 01:11:22.658864 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:22.658842    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjgg2\" (UniqueName: \"kubernetes.io/projected/72009548-a26c-414b-a747-fdb65a2707f8-kube-api-access-sjgg2\") pod \"feast-credit-scoring-fd659f88c-wn5hv\" (UID: \"72009548-a26c-414b-a747-fdb65a2707f8\") " pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"
May 28 01:11:23.153992 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:23.153945    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/72009548-a26c-414b-a747-fdb65a2707f8-online-tls\") pod \"feast-credit-scoring-fd659f88c-wn5hv\" (UID: \"72009548-a26c-414b-a747-fdb65a2707f8\") " pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"
May 28 01:11:23.156369 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:23.156344    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/72009548-a26c-414b-a747-fdb65a2707f8-online-tls\") pod \"feast-credit-scoring-fd659f88c-wn5hv\" (UID: \"72009548-a26c-414b-a747-fdb65a2707f8\") " pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"
May 28 01:11:23.353554 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:23.353515    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"
May 28 01:11:23.480689 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:23.480657    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"]
May 28 01:11:23.483773 ip-10-0-131-73 kubenswrapper[2550]: W0528 01:11:23.483750    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72009548_a26c_414b_a747_fdb65a2707f8.slice/crio-c24281388c49c76fe924f7606cfa447fb176007eff5adf41ec81c02e823f7419 WatchSource:0}: Error finding container c24281388c49c76fe924f7606cfa447fb176007eff5adf41ec81c02e823f7419: Status 404 returned error can't find the container with id c24281388c49c76fe924f7606cfa447fb176007eff5adf41ec81c02e823f7419
May 28 01:11:24.204961 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:24.204921    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv" event={"ID":"72009548-a26c-414b-a747-fdb65a2707f8","Type":"ContainerStarted","Data":"7704d74b390b9ebbba0d3a66595628f60695c130aa93324364f74e5e79de0374"}
May 28 01:11:24.205136 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:24.204969    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv" event={"ID":"72009548-a26c-414b-a747-fdb65a2707f8","Type":"ContainerStarted","Data":"c24281388c49c76fe924f7606cfa447fb176007eff5adf41ec81c02e823f7419"}
May 28 01:11:25.208379 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:25.208297    2550 generic.go:358] "Generic (PLEG): container finished" podID="72009548-a26c-414b-a747-fdb65a2707f8" containerID="7704d74b390b9ebbba0d3a66595628f60695c130aa93324364f74e5e79de0374" exitCode=0
May 28 01:11:25.208735 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:25.208381    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv" event={"ID":"72009548-a26c-414b-a747-fdb65a2707f8","Type":"ContainerDied","Data":"7704d74b390b9ebbba0d3a66595628f60695c130aa93324364f74e5e79de0374"}
May 28 01:11:26.213460 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:26.213401    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv" event={"ID":"72009548-a26c-414b-a747-fdb65a2707f8","Type":"ContainerStarted","Data":"42accaa16426049566df16425328a3e81541c57a336b7e8cfde1ffed68631a0c"}
May 28 01:11:33.234252 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:33.234216    2550 generic.go:358] "Generic (PLEG): container finished" podID="72009548-a26c-414b-a747-fdb65a2707f8" containerID="42accaa16426049566df16425328a3e81541c57a336b7e8cfde1ffed68631a0c" exitCode=0
May 28 01:11:33.234626 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:33.234260    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv" event={"ID":"72009548-a26c-414b-a747-fdb65a2707f8","Type":"ContainerDied","Data":"42accaa16426049566df16425328a3e81541c57a336b7e8cfde1ffed68631a0c"}
May 28 01:11:34.242466 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:34.242405    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv" event={"ID":"72009548-a26c-414b-a747-fdb65a2707f8","Type":"ContainerStarted","Data":"85ff1a277536d67fc104186a04bf9df23b690c953316789da2e7852b2883aae0"}
May 28 01:11:34.242466 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:34.242473    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv" event={"ID":"72009548-a26c-414b-a747-fdb65a2707f8","Type":"ContainerStarted","Data":"2184b0226bcf286978b24ef55ab1ce5b484558459bfbb39770506155f2975c2e"}
May 28 01:11:34.263141 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:34.263090    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv" podStartSLOduration=12.263075024 podStartE2EDuration="12.263075024s" podCreationTimestamp="2026-05-28 01:11:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 01:11:34.261990613 +0000 UTC m=+953.542947707" watchObservedRunningTime="2026-05-28 01:11:34.263075024 +0000 UTC m=+953.544032117"
May 28 01:11:35.353743 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:35.353706    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"
May 28 01:11:35.356144 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:35.353755    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"
May 28 01:11:35.356144 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:35.355368    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv" podUID="72009548-a26c-414b-a747-fdb65a2707f8" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.27:6571: connect: connection refused"
May 28 01:11:35.356144 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:35.355432    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv" podUID="72009548-a26c-414b-a747-fdb65a2707f8" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused"
May 28 01:11:38.354856 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:38.354811    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv" podUID="72009548-a26c-414b-a747-fdb65a2707f8" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused"
May 28 01:11:38.354856 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:38.354825    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv" podUID="72009548-a26c-414b-a747-fdb65a2707f8" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.27:6571: connect: connection refused"
May 28 01:11:41.353975 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:41.353934    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv" podUID="72009548-a26c-414b-a747-fdb65a2707f8" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.27:6571: connect: connection refused"
May 28 01:11:41.361327 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:41.361298    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"
May 28 01:11:41.361791 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:41.361768    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"
May 28 01:11:41.365567 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:41.365538    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"
May 28 01:11:43.354126 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:43.354092    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"
May 28 01:11:44.354770 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:44.354737    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"
May 28 01:11:45.276563 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:45.276535    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"
May 28 01:11:48.299037 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:48.298969    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"]
May 28 01:11:48.303757 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:48.303732    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"
May 28 01:11:48.305945 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:48.305921    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-k9wbg\""
May 28 01:11:48.306040 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:48.305966    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\""
May 28 01:11:48.306085 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:48.306069    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\""
May 28 01:11:48.306128 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:48.306101    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\""
May 28 01:11:48.311827 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:48.311801    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"]
May 28 01:11:48.483410 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:48.483362    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2zx8\" (UniqueName: \"kubernetes.io/projected/bab2d70b-07bc-4f26-a883-f3fc47457f9d-kube-api-access-s2zx8\") pod \"feast-driver-ranking-544b4d576-x4jc9\" (UID: \"bab2d70b-07bc-4f26-a883-f3fc47457f9d\") " pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"
May 28 01:11:48.483606 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:48.483440    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bab2d70b-07bc-4f26-a883-f3fc47457f9d-online-tls\") pod \"feast-driver-ranking-544b4d576-x4jc9\" (UID: \"bab2d70b-07bc-4f26-a883-f3fc47457f9d\") " pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"
May 28 01:11:48.483606 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:48.483466    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bab2d70b-07bc-4f26-a883-f3fc47457f9d-feast-data\") pod \"feast-driver-ranking-544b4d576-x4jc9\" (UID: \"bab2d70b-07bc-4f26-a883-f3fc47457f9d\") " pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"
May 28 01:11:48.483606 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:48.483516    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bab2d70b-07bc-4f26-a883-f3fc47457f9d-registry-tls\") pod \"feast-driver-ranking-544b4d576-x4jc9\" (UID: \"bab2d70b-07bc-4f26-a883-f3fc47457f9d\") " pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"
May 28 01:11:48.584368 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:48.584277    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bab2d70b-07bc-4f26-a883-f3fc47457f9d-registry-tls\") pod \"feast-driver-ranking-544b4d576-x4jc9\" (UID: \"bab2d70b-07bc-4f26-a883-f3fc47457f9d\") " pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"
May 28 01:11:48.584368 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:48.584340    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s2zx8\" (UniqueName: \"kubernetes.io/projected/bab2d70b-07bc-4f26-a883-f3fc47457f9d-kube-api-access-s2zx8\") pod \"feast-driver-ranking-544b4d576-x4jc9\" (UID: \"bab2d70b-07bc-4f26-a883-f3fc47457f9d\") " pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"
May 28 01:11:48.584595 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:48.584377    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bab2d70b-07bc-4f26-a883-f3fc47457f9d-online-tls\") pod \"feast-driver-ranking-544b4d576-x4jc9\" (UID: \"bab2d70b-07bc-4f26-a883-f3fc47457f9d\") " pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"
May 28 01:11:48.584595 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:48.584403    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bab2d70b-07bc-4f26-a883-f3fc47457f9d-feast-data\") pod \"feast-driver-ranking-544b4d576-x4jc9\" (UID: \"bab2d70b-07bc-4f26-a883-f3fc47457f9d\") " pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"
May 28 01:11:48.584595 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:11:48.584454    2550 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-driver-ranking-registry-tls: secret "feast-driver-ranking-registry-tls" not found
May 28 01:11:48.584595 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:11:48.584525    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bab2d70b-07bc-4f26-a883-f3fc47457f9d-registry-tls podName:bab2d70b-07bc-4f26-a883-f3fc47457f9d nodeName:}" failed. No retries permitted until 2026-05-28 01:11:49.084507674 +0000 UTC m=+968.365464746 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/bab2d70b-07bc-4f26-a883-f3fc47457f9d-registry-tls") pod "feast-driver-ranking-544b4d576-x4jc9" (UID: "bab2d70b-07bc-4f26-a883-f3fc47457f9d") : secret "feast-driver-ranking-registry-tls" not found
May 28 01:11:48.584836 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:48.584816    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bab2d70b-07bc-4f26-a883-f3fc47457f9d-feast-data\") pod \"feast-driver-ranking-544b4d576-x4jc9\" (UID: \"bab2d70b-07bc-4f26-a883-f3fc47457f9d\") " pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"
May 28 01:11:48.586815 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:48.586794    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bab2d70b-07bc-4f26-a883-f3fc47457f9d-online-tls\") pod \"feast-driver-ranking-544b4d576-x4jc9\" (UID: \"bab2d70b-07bc-4f26-a883-f3fc47457f9d\") " pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"
May 28 01:11:48.593233 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:48.593206    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2zx8\" (UniqueName: \"kubernetes.io/projected/bab2d70b-07bc-4f26-a883-f3fc47457f9d-kube-api-access-s2zx8\") pod \"feast-driver-ranking-544b4d576-x4jc9\" (UID: \"bab2d70b-07bc-4f26-a883-f3fc47457f9d\") " pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"
May 28 01:11:49.089071 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:49.089039    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bab2d70b-07bc-4f26-a883-f3fc47457f9d-registry-tls\") pod \"feast-driver-ranking-544b4d576-x4jc9\" (UID: \"bab2d70b-07bc-4f26-a883-f3fc47457f9d\") " pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"
May 28 01:11:49.091432 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:49.091395    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bab2d70b-07bc-4f26-a883-f3fc47457f9d-registry-tls\") pod \"feast-driver-ranking-544b4d576-x4jc9\" (UID: \"bab2d70b-07bc-4f26-a883-f3fc47457f9d\") " pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"
May 28 01:11:49.213066 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:49.213020    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"
May 28 01:11:49.349074 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:49.349046    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"]
May 28 01:11:49.351520 ip-10-0-131-73 kubenswrapper[2550]: W0528 01:11:49.351490    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbab2d70b_07bc_4f26_a883_f3fc47457f9d.slice/crio-24e4ab45892b5e505e7bdb0bedf1171dbe1b86ad948d222b247260abb4bded83 WatchSource:0}: Error finding container 24e4ab45892b5e505e7bdb0bedf1171dbe1b86ad948d222b247260abb4bded83: Status 404 returned error can't find the container with id 24e4ab45892b5e505e7bdb0bedf1171dbe1b86ad948d222b247260abb4bded83
May 28 01:11:50.291581 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:50.291543    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9" event={"ID":"bab2d70b-07bc-4f26-a883-f3fc47457f9d","Type":"ContainerStarted","Data":"79de74eaaf772ed63b7f25ed9910b1099337e207ee886a6ddf6e7c8b392cae96"}
May 28 01:11:50.291819 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:50.291803    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9" event={"ID":"bab2d70b-07bc-4f26-a883-f3fc47457f9d","Type":"ContainerStarted","Data":"24e4ab45892b5e505e7bdb0bedf1171dbe1b86ad948d222b247260abb4bded83"}
May 28 01:11:54.303789 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:54.303698    2550 generic.go:358] "Generic (PLEG): container finished" podID="bab2d70b-07bc-4f26-a883-f3fc47457f9d" containerID="79de74eaaf772ed63b7f25ed9910b1099337e207ee886a6ddf6e7c8b392cae96" exitCode=0
May 28 01:11:54.304131 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:54.303777    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9" event={"ID":"bab2d70b-07bc-4f26-a883-f3fc47457f9d","Type":"ContainerDied","Data":"79de74eaaf772ed63b7f25ed9910b1099337e207ee886a6ddf6e7c8b392cae96"}
May 28 01:11:55.308088 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:11:55.308054    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9" event={"ID":"bab2d70b-07bc-4f26-a883-f3fc47457f9d","Type":"ContainerStarted","Data":"022d53a3bf2abca339e4e52acbdc9922819f5623e8d22b435823d142d19a1b56"}
May 28 01:12:02.330299 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:12:02.330263    2550 generic.go:358] "Generic (PLEG): container finished" podID="bab2d70b-07bc-4f26-a883-f3fc47457f9d" containerID="022d53a3bf2abca339e4e52acbdc9922819f5623e8d22b435823d142d19a1b56" exitCode=0
May 28 01:12:02.330676 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:12:02.330331    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9" event={"ID":"bab2d70b-07bc-4f26-a883-f3fc47457f9d","Type":"ContainerDied","Data":"022d53a3bf2abca339e4e52acbdc9922819f5623e8d22b435823d142d19a1b56"}
May 28 01:12:03.335900 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:12:03.335865    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9" event={"ID":"bab2d70b-07bc-4f26-a883-f3fc47457f9d","Type":"ContainerStarted","Data":"ffb0ab84118ee9922b0c32694af894f4a6c930046cb48c5f2e9f72d87be938e5"}
May 28 01:12:03.335900 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:12:03.335907    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9" event={"ID":"bab2d70b-07bc-4f26-a883-f3fc47457f9d","Type":"ContainerStarted","Data":"2d6ca6f669bb3716e99e3e0b6abc09c37689b544111a9c0f3ad4a36c22fa060e"}
May 28 01:12:03.359010 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:12:03.358964    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9" podStartSLOduration=15.358949727 podStartE2EDuration="15.358949727s" podCreationTimestamp="2026-05-28 01:11:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 01:12:03.356723303 +0000 UTC m=+982.637680397" watchObservedRunningTime="2026-05-28 01:12:03.358949727 +0000 UTC m=+982.639906821"
May 28 01:12:04.213759 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:12:04.213723    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"
May 28 01:12:04.213950 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:12:04.213773    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"
May 28 01:12:04.215500 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:12:04.215459    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9" podUID="bab2d70b-07bc-4f26-a883-f3fc47457f9d" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.28:6571: connect: connection refused"
May 28 01:12:04.215500 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:12:04.215484    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9" podUID="bab2d70b-07bc-4f26-a883-f3fc47457f9d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused"
May 28 01:12:07.213693 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:12:07.213642    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9" podUID="bab2d70b-07bc-4f26-a883-f3fc47457f9d" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.28:6571: connect: connection refused"
May 28 01:12:07.214162 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:12:07.213730    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9" podUID="bab2d70b-07bc-4f26-a883-f3fc47457f9d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused"
May 28 01:12:09.213720 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:12:09.213674    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"
May 28 01:12:09.214089 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:12:09.213734    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"
May 28 01:12:10.213766 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:12:10.213718    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9" podUID="bab2d70b-07bc-4f26-a883-f3fc47457f9d" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.28:6571: connect: connection refused"
May 28 01:12:10.219128 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:12:10.219098    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"
May 28 01:12:10.222321 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:12:10.222298    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"
May 28 01:12:13.214004 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:12:13.213954    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9" podUID="bab2d70b-07bc-4f26-a883-f3fc47457f9d" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.28:6571: connect: connection refused"
May 28 01:12:16.214258 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:12:16.214225    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"
May 28 01:12:16.215059 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:12:16.215038    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"
May 28 01:13:32.252799 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:32.252719    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5d796b8684-jqqqx"]
May 28 01:13:32.253301 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:32.252975    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-5d796b8684-jqqqx" podUID="0fe53fb5-edc8-47fe-9a2c-b1d076959f64" containerName="manager" containerID="cri-o://be234907ccdfa351764a14c3925fc89820afe3e7406783d24214d4e302aaa053" gracePeriod=10
May 28 01:13:32.490605 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:32.490565    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"]
May 28 01:13:32.490966 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:32.490938    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv" podUID="72009548-a26c-414b-a747-fdb65a2707f8" containerName="registry" containerID="cri-o://2184b0226bcf286978b24ef55ab1ce5b484558459bfbb39770506155f2975c2e" gracePeriod=30
May 28 01:13:32.491123 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:32.491032    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv" podUID="72009548-a26c-414b-a747-fdb65a2707f8" containerName="online" containerID="cri-o://85ff1a277536d67fc104186a04bf9df23b690c953316789da2e7852b2883aae0" gracePeriod=30
May 28 01:13:32.501029 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:32.501002    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"]
May 28 01:13:32.501308 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:32.501280    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9" podUID="bab2d70b-07bc-4f26-a883-f3fc47457f9d" containerName="registry" containerID="cri-o://2d6ca6f669bb3716e99e3e0b6abc09c37689b544111a9c0f3ad4a36c22fa060e" gracePeriod=30
May 28 01:13:32.501387 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:32.501321    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9" podUID="bab2d70b-07bc-4f26-a883-f3fc47457f9d" containerName="online" containerID="cri-o://ffb0ab84118ee9922b0c32694af894f4a6c930046cb48c5f2e9f72d87be938e5" gracePeriod=30
May 28 01:13:32.537227 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:32.537207    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5d796b8684-jqqqx"
May 28 01:13:32.574669 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:32.574632    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8w8w\" (UniqueName: \"kubernetes.io/projected/0fe53fb5-edc8-47fe-9a2c-b1d076959f64-kube-api-access-t8w8w\") pod \"0fe53fb5-edc8-47fe-9a2c-b1d076959f64\" (UID: \"0fe53fb5-edc8-47fe-9a2c-b1d076959f64\") "
May 28 01:13:32.576782 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:32.576757    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fe53fb5-edc8-47fe-9a2c-b1d076959f64-kube-api-access-t8w8w" (OuterVolumeSpecName: "kube-api-access-t8w8w") pod "0fe53fb5-edc8-47fe-9a2c-b1d076959f64" (UID: "0fe53fb5-edc8-47fe-9a2c-b1d076959f64"). InnerVolumeSpecName "kube-api-access-t8w8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 01:13:32.593993 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:32.593962    2550 generic.go:358] "Generic (PLEG): container finished" podID="0fe53fb5-edc8-47fe-9a2c-b1d076959f64" containerID="be234907ccdfa351764a14c3925fc89820afe3e7406783d24214d4e302aaa053" exitCode=0
May 28 01:13:32.594089 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:32.594031    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5d796b8684-jqqqx"
May 28 01:13:32.594089 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:32.594038    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5d796b8684-jqqqx" event={"ID":"0fe53fb5-edc8-47fe-9a2c-b1d076959f64","Type":"ContainerDied","Data":"be234907ccdfa351764a14c3925fc89820afe3e7406783d24214d4e302aaa053"}
May 28 01:13:32.594089 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:32.594075    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5d796b8684-jqqqx" event={"ID":"0fe53fb5-edc8-47fe-9a2c-b1d076959f64","Type":"ContainerDied","Data":"7f734598383b09812c2ed7edca3990dd4dbb6bf4fd39ef1f53767c4d8b2f0586"}
May 28 01:13:32.594089 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:32.594089    2550 scope.go:117] "RemoveContainer" containerID="be234907ccdfa351764a14c3925fc89820afe3e7406783d24214d4e302aaa053"
May 28 01:13:32.602361 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:32.602339    2550 scope.go:117] "RemoveContainer" containerID="be234907ccdfa351764a14c3925fc89820afe3e7406783d24214d4e302aaa053"
May 28 01:13:32.602686 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:13:32.602670    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be234907ccdfa351764a14c3925fc89820afe3e7406783d24214d4e302aaa053\": container with ID starting with be234907ccdfa351764a14c3925fc89820afe3e7406783d24214d4e302aaa053 not found: ID does not exist" containerID="be234907ccdfa351764a14c3925fc89820afe3e7406783d24214d4e302aaa053"
May 28 01:13:32.602748 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:32.602694    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be234907ccdfa351764a14c3925fc89820afe3e7406783d24214d4e302aaa053"} err="failed to get container status \"be234907ccdfa351764a14c3925fc89820afe3e7406783d24214d4e302aaa053\": rpc error: code = NotFound desc = could not find container \"be234907ccdfa351764a14c3925fc89820afe3e7406783d24214d4e302aaa053\": container with ID starting with be234907ccdfa351764a14c3925fc89820afe3e7406783d24214d4e302aaa053 not found: ID does not exist"
May 28 01:13:32.613752 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:32.613727    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5d796b8684-jqqqx"]
May 28 01:13:32.617719 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:32.617696    2550 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5d796b8684-jqqqx"]
May 28 01:13:32.675407 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:32.675374    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8w8w\" (UniqueName: \"kubernetes.io/projected/0fe53fb5-edc8-47fe-9a2c-b1d076959f64-kube-api-access-t8w8w\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:13:33.324274 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:33.324248    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fe53fb5-edc8-47fe-9a2c-b1d076959f64" path="/var/lib/kubelet/pods/0fe53fb5-edc8-47fe-9a2c-b1d076959f64/volumes"
May 28 01:13:33.602112 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:33.602033    2550 generic.go:358] "Generic (PLEG): container finished" podID="bab2d70b-07bc-4f26-a883-f3fc47457f9d" containerID="ffb0ab84118ee9922b0c32694af894f4a6c930046cb48c5f2e9f72d87be938e5" exitCode=0
May 28 01:13:33.602279 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:33.602103    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9" event={"ID":"bab2d70b-07bc-4f26-a883-f3fc47457f9d","Type":"ContainerDied","Data":"ffb0ab84118ee9922b0c32694af894f4a6c930046cb48c5f2e9f72d87be938e5"}
May 28 01:13:33.604037 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:33.604017    2550 generic.go:358] "Generic (PLEG): container finished" podID="72009548-a26c-414b-a747-fdb65a2707f8" containerID="85ff1a277536d67fc104186a04bf9df23b690c953316789da2e7852b2883aae0" exitCode=0
May 28 01:13:33.604156 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:33.604063    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv" event={"ID":"72009548-a26c-414b-a747-fdb65a2707f8","Type":"ContainerDied","Data":"85ff1a277536d67fc104186a04bf9df23b690c953316789da2e7852b2883aae0"}
May 28 01:13:34.919308 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:34.919272    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-bgkbm"]
May 28 01:13:34.919781 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:34.919504    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-7c7656cc6d-bgkbm" podUID="186635cb-ee5f-4051-9b3f-4d2228e1e048" containerName="postgres" containerID="cri-o://718008af9c82375013882ddf420f039e147f6ae80b45e29bcadeefc2044c2675" gracePeriod=30
May 28 01:13:34.923748 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:34.923720    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-854544c579-6ssbx"]
May 28 01:13:34.923906 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:34.923887    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/redis-854544c579-6ssbx" podUID="664b6570-2c2e-4260-b2a9-5425eeddbec1" containerName="redis" containerID="cri-o://17fd31d37bd954aeb2e98cb7590fe962480e3841f1fd2aa172b037201eae17d1" gracePeriod=30
May 28 01:13:35.163241 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.163218    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-bgkbm"
May 28 01:13:35.196984 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.196836    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/186635cb-ee5f-4051-9b3f-4d2228e1e048-postgresdata\") pod \"186635cb-ee5f-4051-9b3f-4d2228e1e048\" (UID: \"186635cb-ee5f-4051-9b3f-4d2228e1e048\") "
May 28 01:13:35.196984 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.196883    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4hqx\" (UniqueName: \"kubernetes.io/projected/186635cb-ee5f-4051-9b3f-4d2228e1e048-kube-api-access-k4hqx\") pod \"186635cb-ee5f-4051-9b3f-4d2228e1e048\" (UID: \"186635cb-ee5f-4051-9b3f-4d2228e1e048\") "
May 28 01:13:35.200572 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.200522    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/186635cb-ee5f-4051-9b3f-4d2228e1e048-kube-api-access-k4hqx" (OuterVolumeSpecName: "kube-api-access-k4hqx") pod "186635cb-ee5f-4051-9b3f-4d2228e1e048" (UID: "186635cb-ee5f-4051-9b3f-4d2228e1e048"). InnerVolumeSpecName "kube-api-access-k4hqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 01:13:35.230430 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.230378    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/186635cb-ee5f-4051-9b3f-4d2228e1e048-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "186635cb-ee5f-4051-9b3f-4d2228e1e048" (UID: "186635cb-ee5f-4051-9b3f-4d2228e1e048"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 01:13:35.276270 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.276246    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-6ssbx"
May 28 01:13:35.298096 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.298068    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zw9jf\" (UniqueName: \"kubernetes.io/projected/664b6570-2c2e-4260-b2a9-5425eeddbec1-kube-api-access-zw9jf\") pod \"664b6570-2c2e-4260-b2a9-5425eeddbec1\" (UID: \"664b6570-2c2e-4260-b2a9-5425eeddbec1\") "
May 28 01:13:35.298305 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.298290    2550 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/186635cb-ee5f-4051-9b3f-4d2228e1e048-postgresdata\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:13:35.298386 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.298307    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4hqx\" (UniqueName: \"kubernetes.io/projected/186635cb-ee5f-4051-9b3f-4d2228e1e048-kube-api-access-k4hqx\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:13:35.300239 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.300205    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/664b6570-2c2e-4260-b2a9-5425eeddbec1-kube-api-access-zw9jf" (OuterVolumeSpecName: "kube-api-access-zw9jf") pod "664b6570-2c2e-4260-b2a9-5425eeddbec1" (UID: "664b6570-2c2e-4260-b2a9-5425eeddbec1"). InnerVolumeSpecName "kube-api-access-zw9jf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 01:13:35.399296 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.399263    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zw9jf\" (UniqueName: \"kubernetes.io/projected/664b6570-2c2e-4260-b2a9-5425eeddbec1-kube-api-access-zw9jf\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:13:35.610824 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.610730    2550 generic.go:358] "Generic (PLEG): container finished" podID="664b6570-2c2e-4260-b2a9-5425eeddbec1" containerID="17fd31d37bd954aeb2e98cb7590fe962480e3841f1fd2aa172b037201eae17d1" exitCode=0
May 28 01:13:35.610824 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.610777    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-6ssbx"
May 28 01:13:35.611057 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.610818    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-6ssbx" event={"ID":"664b6570-2c2e-4260-b2a9-5425eeddbec1","Type":"ContainerDied","Data":"17fd31d37bd954aeb2e98cb7590fe962480e3841f1fd2aa172b037201eae17d1"}
May 28 01:13:35.611057 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.610856    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-6ssbx" event={"ID":"664b6570-2c2e-4260-b2a9-5425eeddbec1","Type":"ContainerDied","Data":"b771f6ac6027a1eeba46533f5e81472fba05e4c14d6a82f6537dd35f9d996690"}
May 28 01:13:35.611057 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.610875    2550 scope.go:117] "RemoveContainer" containerID="17fd31d37bd954aeb2e98cb7590fe962480e3841f1fd2aa172b037201eae17d1"
May 28 01:13:35.611982 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.611962    2550 generic.go:358] "Generic (PLEG): container finished" podID="186635cb-ee5f-4051-9b3f-4d2228e1e048" containerID="718008af9c82375013882ddf420f039e147f6ae80b45e29bcadeefc2044c2675" exitCode=0
May 28 01:13:35.612073 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.611996    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-bgkbm" event={"ID":"186635cb-ee5f-4051-9b3f-4d2228e1e048","Type":"ContainerDied","Data":"718008af9c82375013882ddf420f039e147f6ae80b45e29bcadeefc2044c2675"}
May 28 01:13:35.612073 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.612025    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-bgkbm"
May 28 01:13:35.612073 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.612039    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-bgkbm" event={"ID":"186635cb-ee5f-4051-9b3f-4d2228e1e048","Type":"ContainerDied","Data":"4378ed20d9238924bbec88cbc1a9f0843987b1bfbf00ba83131b41976dfa5a97"}
May 28 01:13:35.618234 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.618213    2550 scope.go:117] "RemoveContainer" containerID="17fd31d37bd954aeb2e98cb7590fe962480e3841f1fd2aa172b037201eae17d1"
May 28 01:13:35.618506 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:13:35.618487    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17fd31d37bd954aeb2e98cb7590fe962480e3841f1fd2aa172b037201eae17d1\": container with ID starting with 17fd31d37bd954aeb2e98cb7590fe962480e3841f1fd2aa172b037201eae17d1 not found: ID does not exist" containerID="17fd31d37bd954aeb2e98cb7590fe962480e3841f1fd2aa172b037201eae17d1"
May 28 01:13:35.618588 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.618518    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17fd31d37bd954aeb2e98cb7590fe962480e3841f1fd2aa172b037201eae17d1"} err="failed to get container status \"17fd31d37bd954aeb2e98cb7590fe962480e3841f1fd2aa172b037201eae17d1\": rpc error: code = NotFound desc = could not find container \"17fd31d37bd954aeb2e98cb7590fe962480e3841f1fd2aa172b037201eae17d1\": container with ID starting with 17fd31d37bd954aeb2e98cb7590fe962480e3841f1fd2aa172b037201eae17d1 not found: ID does not exist"
May 28 01:13:35.618588 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.618554    2550 scope.go:117] "RemoveContainer" containerID="718008af9c82375013882ddf420f039e147f6ae80b45e29bcadeefc2044c2675"
May 28 01:13:35.624996 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.624978    2550 scope.go:117] "RemoveContainer" containerID="718008af9c82375013882ddf420f039e147f6ae80b45e29bcadeefc2044c2675"
May 28 01:13:35.625321 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:13:35.625293    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"718008af9c82375013882ddf420f039e147f6ae80b45e29bcadeefc2044c2675\": container with ID starting with 718008af9c82375013882ddf420f039e147f6ae80b45e29bcadeefc2044c2675 not found: ID does not exist" containerID="718008af9c82375013882ddf420f039e147f6ae80b45e29bcadeefc2044c2675"
May 28 01:13:35.625608 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.625322    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"718008af9c82375013882ddf420f039e147f6ae80b45e29bcadeefc2044c2675"} err="failed to get container status \"718008af9c82375013882ddf420f039e147f6ae80b45e29bcadeefc2044c2675\": rpc error: code = NotFound desc = could not find container \"718008af9c82375013882ddf420f039e147f6ae80b45e29bcadeefc2044c2675\": container with ID starting with 718008af9c82375013882ddf420f039e147f6ae80b45e29bcadeefc2044c2675 not found: ID does not exist"
May 28 01:13:35.626956 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.626934    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-bgkbm"]
May 28 01:13:35.629846 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.629827    2550 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-bgkbm"]
May 28 01:13:35.638450 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.638426    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-854544c579-6ssbx"]
May 28 01:13:35.641597 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:35.641580    2550 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-854544c579-6ssbx"]
May 28 01:13:37.323253 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:37.323208    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="186635cb-ee5f-4051-9b3f-4d2228e1e048" path="/var/lib/kubelet/pods/186635cb-ee5f-4051-9b3f-4d2228e1e048/volumes"
May 28 01:13:37.323744 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:37.323689    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="664b6570-2c2e-4260-b2a9-5425eeddbec1" path="/var/lib/kubelet/pods/664b6570-2c2e-4260-b2a9-5425eeddbec1/volumes"
May 28 01:13:40.220105 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:40.220054    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9" podUID="bab2d70b-07bc-4f26-a883-f3fc47457f9d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused"
May 28 01:13:41.362286 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:41.362242    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv" podUID="72009548-a26c-414b-a747-fdb65a2707f8" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused"
May 28 01:13:50.220523 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:50.220481    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9" podUID="bab2d70b-07bc-4f26-a883-f3fc47457f9d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused"
May 28 01:13:51.362321 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:13:51.362277    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv" podUID="72009548-a26c-414b-a747-fdb65a2707f8" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused"
May 28 01:14:00.220117 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:00.220064    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9" podUID="bab2d70b-07bc-4f26-a883-f3fc47457f9d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused"
May 28 01:14:00.220514 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:00.220222    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"
May 28 01:14:01.362849 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:01.362804    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv" podUID="72009548-a26c-414b-a747-fdb65a2707f8" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused"
May 28 01:14:01.363224 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:01.362926    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"
May 28 01:14:02.690132 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.690110    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"
May 28 01:14:02.693939 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.693912    2550 generic.go:358] "Generic (PLEG): container finished" podID="72009548-a26c-414b-a747-fdb65a2707f8" containerID="2184b0226bcf286978b24ef55ab1ce5b484558459bfbb39770506155f2975c2e" exitCode=137
May 28 01:14:02.694062 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.693998    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv" event={"ID":"72009548-a26c-414b-a747-fdb65a2707f8","Type":"ContainerDied","Data":"2184b0226bcf286978b24ef55ab1ce5b484558459bfbb39770506155f2975c2e"}
May 28 01:14:02.695831 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.695809    2550 generic.go:358] "Generic (PLEG): container finished" podID="bab2d70b-07bc-4f26-a883-f3fc47457f9d" containerID="2d6ca6f669bb3716e99e3e0b6abc09c37689b544111a9c0f3ad4a36c22fa060e" exitCode=137
May 28 01:14:02.695921 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.695843    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9" event={"ID":"bab2d70b-07bc-4f26-a883-f3fc47457f9d","Type":"ContainerDied","Data":"2d6ca6f669bb3716e99e3e0b6abc09c37689b544111a9c0f3ad4a36c22fa060e"}
May 28 01:14:02.695921 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.695860    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9" event={"ID":"bab2d70b-07bc-4f26-a883-f3fc47457f9d","Type":"ContainerDied","Data":"24e4ab45892b5e505e7bdb0bedf1171dbe1b86ad948d222b247260abb4bded83"}
May 28 01:14:02.695921 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.695875    2550 scope.go:117] "RemoveContainer" containerID="ffb0ab84118ee9922b0c32694af894f4a6c930046cb48c5f2e9f72d87be938e5"
May 28 01:14:02.695921 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.695875    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"
May 28 01:14:02.702934 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.702916    2550 scope.go:117] "RemoveContainer" containerID="2d6ca6f669bb3716e99e3e0b6abc09c37689b544111a9c0f3ad4a36c22fa060e"
May 28 01:14:02.711140 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.711098    2550 scope.go:117] "RemoveContainer" containerID="022d53a3bf2abca339e4e52acbdc9922819f5623e8d22b435823d142d19a1b56"
May 28 01:14:02.719959 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.719940    2550 scope.go:117] "RemoveContainer" containerID="79de74eaaf772ed63b7f25ed9910b1099337e207ee886a6ddf6e7c8b392cae96"
May 28 01:14:02.728751 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.728721    2550 scope.go:117] "RemoveContainer" containerID="ffb0ab84118ee9922b0c32694af894f4a6c930046cb48c5f2e9f72d87be938e5"
May 28 01:14:02.729200 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:14:02.729174    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffb0ab84118ee9922b0c32694af894f4a6c930046cb48c5f2e9f72d87be938e5\": container with ID starting with ffb0ab84118ee9922b0c32694af894f4a6c930046cb48c5f2e9f72d87be938e5 not found: ID does not exist" containerID="ffb0ab84118ee9922b0c32694af894f4a6c930046cb48c5f2e9f72d87be938e5"
May 28 01:14:02.729265 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.729204    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffb0ab84118ee9922b0c32694af894f4a6c930046cb48c5f2e9f72d87be938e5"} err="failed to get container status \"ffb0ab84118ee9922b0c32694af894f4a6c930046cb48c5f2e9f72d87be938e5\": rpc error: code = NotFound desc = could not find container \"ffb0ab84118ee9922b0c32694af894f4a6c930046cb48c5f2e9f72d87be938e5\": container with ID starting with ffb0ab84118ee9922b0c32694af894f4a6c930046cb48c5f2e9f72d87be938e5 not found: ID does not exist"
May 28 01:14:02.729265 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.729226    2550 scope.go:117] "RemoveContainer" containerID="2d6ca6f669bb3716e99e3e0b6abc09c37689b544111a9c0f3ad4a36c22fa060e"
May 28 01:14:02.729542 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:14:02.729523    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d6ca6f669bb3716e99e3e0b6abc09c37689b544111a9c0f3ad4a36c22fa060e\": container with ID starting with 2d6ca6f669bb3716e99e3e0b6abc09c37689b544111a9c0f3ad4a36c22fa060e not found: ID does not exist" containerID="2d6ca6f669bb3716e99e3e0b6abc09c37689b544111a9c0f3ad4a36c22fa060e"
May 28 01:14:02.729617 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.729549    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d6ca6f669bb3716e99e3e0b6abc09c37689b544111a9c0f3ad4a36c22fa060e"} err="failed to get container status \"2d6ca6f669bb3716e99e3e0b6abc09c37689b544111a9c0f3ad4a36c22fa060e\": rpc error: code = NotFound desc = could not find container \"2d6ca6f669bb3716e99e3e0b6abc09c37689b544111a9c0f3ad4a36c22fa060e\": container with ID starting with 2d6ca6f669bb3716e99e3e0b6abc09c37689b544111a9c0f3ad4a36c22fa060e not found: ID does not exist"
May 28 01:14:02.729617 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.729565    2550 scope.go:117] "RemoveContainer" containerID="022d53a3bf2abca339e4e52acbdc9922819f5623e8d22b435823d142d19a1b56"
May 28 01:14:02.729806 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:14:02.729789    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"022d53a3bf2abca339e4e52acbdc9922819f5623e8d22b435823d142d19a1b56\": container with ID starting with 022d53a3bf2abca339e4e52acbdc9922819f5623e8d22b435823d142d19a1b56 not found: ID does not exist" containerID="022d53a3bf2abca339e4e52acbdc9922819f5623e8d22b435823d142d19a1b56"
May 28 01:14:02.729855 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.729810    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"022d53a3bf2abca339e4e52acbdc9922819f5623e8d22b435823d142d19a1b56"} err="failed to get container status \"022d53a3bf2abca339e4e52acbdc9922819f5623e8d22b435823d142d19a1b56\": rpc error: code = NotFound desc = could not find container \"022d53a3bf2abca339e4e52acbdc9922819f5623e8d22b435823d142d19a1b56\": container with ID starting with 022d53a3bf2abca339e4e52acbdc9922819f5623e8d22b435823d142d19a1b56 not found: ID does not exist"
May 28 01:14:02.729855 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.729826    2550 scope.go:117] "RemoveContainer" containerID="79de74eaaf772ed63b7f25ed9910b1099337e207ee886a6ddf6e7c8b392cae96"
May 28 01:14:02.730036 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:14:02.730022    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79de74eaaf772ed63b7f25ed9910b1099337e207ee886a6ddf6e7c8b392cae96\": container with ID starting with 79de74eaaf772ed63b7f25ed9910b1099337e207ee886a6ddf6e7c8b392cae96 not found: ID does not exist" containerID="79de74eaaf772ed63b7f25ed9910b1099337e207ee886a6ddf6e7c8b392cae96"
May 28 01:14:02.730086 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.730038    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79de74eaaf772ed63b7f25ed9910b1099337e207ee886a6ddf6e7c8b392cae96"} err="failed to get container status \"79de74eaaf772ed63b7f25ed9910b1099337e207ee886a6ddf6e7c8b392cae96\": rpc error: code = NotFound desc = could not find container \"79de74eaaf772ed63b7f25ed9910b1099337e207ee886a6ddf6e7c8b392cae96\": container with ID starting with 79de74eaaf772ed63b7f25ed9910b1099337e207ee886a6ddf6e7c8b392cae96 not found: ID does not exist"
May 28 01:14:02.820673 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.820591    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bab2d70b-07bc-4f26-a883-f3fc47457f9d-registry-tls\") pod \"bab2d70b-07bc-4f26-a883-f3fc47457f9d\" (UID: \"bab2d70b-07bc-4f26-a883-f3fc47457f9d\") "
May 28 01:14:02.820673 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.820637    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2zx8\" (UniqueName: \"kubernetes.io/projected/bab2d70b-07bc-4f26-a883-f3fc47457f9d-kube-api-access-s2zx8\") pod \"bab2d70b-07bc-4f26-a883-f3fc47457f9d\" (UID: \"bab2d70b-07bc-4f26-a883-f3fc47457f9d\") "
May 28 01:14:02.820673 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.820666    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bab2d70b-07bc-4f26-a883-f3fc47457f9d-online-tls\") pod \"bab2d70b-07bc-4f26-a883-f3fc47457f9d\" (UID: \"bab2d70b-07bc-4f26-a883-f3fc47457f9d\") "
May 28 01:14:02.820865 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.820700    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bab2d70b-07bc-4f26-a883-f3fc47457f9d-feast-data\") pod \"bab2d70b-07bc-4f26-a883-f3fc47457f9d\" (UID: \"bab2d70b-07bc-4f26-a883-f3fc47457f9d\") "
May 28 01:14:02.821252 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.821220    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bab2d70b-07bc-4f26-a883-f3fc47457f9d-feast-data" (OuterVolumeSpecName: "feast-data") pod "bab2d70b-07bc-4f26-a883-f3fc47457f9d" (UID: "bab2d70b-07bc-4f26-a883-f3fc47457f9d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 01:14:02.822727 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.822704    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bab2d70b-07bc-4f26-a883-f3fc47457f9d-online-tls" (OuterVolumeSpecName: "online-tls") pod "bab2d70b-07bc-4f26-a883-f3fc47457f9d" (UID: "bab2d70b-07bc-4f26-a883-f3fc47457f9d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:14:02.822833 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.822731    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bab2d70b-07bc-4f26-a883-f3fc47457f9d-kube-api-access-s2zx8" (OuterVolumeSpecName: "kube-api-access-s2zx8") pod "bab2d70b-07bc-4f26-a883-f3fc47457f9d" (UID: "bab2d70b-07bc-4f26-a883-f3fc47457f9d"). InnerVolumeSpecName "kube-api-access-s2zx8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 01:14:02.822833 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.822792    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bab2d70b-07bc-4f26-a883-f3fc47457f9d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bab2d70b-07bc-4f26-a883-f3fc47457f9d" (UID: "bab2d70b-07bc-4f26-a883-f3fc47457f9d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:14:02.921509 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.921470    2550 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bab2d70b-07bc-4f26-a883-f3fc47457f9d-registry-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:14:02.921509 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.921502    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2zx8\" (UniqueName: \"kubernetes.io/projected/bab2d70b-07bc-4f26-a883-f3fc47457f9d-kube-api-access-s2zx8\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:14:02.921509 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.921514    2550 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bab2d70b-07bc-4f26-a883-f3fc47457f9d-online-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:14:02.921735 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:02.921524    2550 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bab2d70b-07bc-4f26-a883-f3fc47457f9d-feast-data\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:14:03.014465 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:03.014435    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"]
May 28 01:14:03.016804 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:03.016779    2550 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-544b4d576-x4jc9"]
May 28 01:14:03.127350 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:03.127327    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"
May 28 01:14:03.223148 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:03.223106    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/72009548-a26c-414b-a747-fdb65a2707f8-online-tls\") pod \"72009548-a26c-414b-a747-fdb65a2707f8\" (UID: \"72009548-a26c-414b-a747-fdb65a2707f8\") "
May 28 01:14:03.223345 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:03.223171    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjgg2\" (UniqueName: \"kubernetes.io/projected/72009548-a26c-414b-a747-fdb65a2707f8-kube-api-access-sjgg2\") pod \"72009548-a26c-414b-a747-fdb65a2707f8\" (UID: \"72009548-a26c-414b-a747-fdb65a2707f8\") "
May 28 01:14:03.223345 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:03.223234    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/72009548-a26c-414b-a747-fdb65a2707f8-registry-tls\") pod \"72009548-a26c-414b-a747-fdb65a2707f8\" (UID: \"72009548-a26c-414b-a747-fdb65a2707f8\") "
May 28 01:14:03.223345 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:03.223269    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/72009548-a26c-414b-a747-fdb65a2707f8-feast-data\") pod \"72009548-a26c-414b-a747-fdb65a2707f8\" (UID: \"72009548-a26c-414b-a747-fdb65a2707f8\") "
May 28 01:14:03.225175 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:03.225135    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72009548-a26c-414b-a747-fdb65a2707f8-kube-api-access-sjgg2" (OuterVolumeSpecName: "kube-api-access-sjgg2") pod "72009548-a26c-414b-a747-fdb65a2707f8" (UID: "72009548-a26c-414b-a747-fdb65a2707f8"). InnerVolumeSpecName "kube-api-access-sjgg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 01:14:03.225284 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:03.225174    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72009548-a26c-414b-a747-fdb65a2707f8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "72009548-a26c-414b-a747-fdb65a2707f8" (UID: "72009548-a26c-414b-a747-fdb65a2707f8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:14:03.225284 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:03.225193    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72009548-a26c-414b-a747-fdb65a2707f8-online-tls" (OuterVolumeSpecName: "online-tls") pod "72009548-a26c-414b-a747-fdb65a2707f8" (UID: "72009548-a26c-414b-a747-fdb65a2707f8"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:14:03.231872 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:03.231851    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72009548-a26c-414b-a747-fdb65a2707f8-feast-data" (OuterVolumeSpecName: "feast-data") pod "72009548-a26c-414b-a747-fdb65a2707f8" (UID: "72009548-a26c-414b-a747-fdb65a2707f8"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 01:14:03.323702 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:03.323668    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bab2d70b-07bc-4f26-a883-f3fc47457f9d" path="/var/lib/kubelet/pods/bab2d70b-07bc-4f26-a883-f3fc47457f9d/volumes"
May 28 01:14:03.324221 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:03.324204    2550 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/72009548-a26c-414b-a747-fdb65a2707f8-registry-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:14:03.324268 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:03.324224    2550 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/72009548-a26c-414b-a747-fdb65a2707f8-feast-data\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:14:03.324268 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:03.324235    2550 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/72009548-a26c-414b-a747-fdb65a2707f8-online-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:14:03.324268 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:03.324243    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjgg2\" (UniqueName: \"kubernetes.io/projected/72009548-a26c-414b-a747-fdb65a2707f8-kube-api-access-sjgg2\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:14:03.705491 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:03.705455    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv" event={"ID":"72009548-a26c-414b-a747-fdb65a2707f8","Type":"ContainerDied","Data":"c24281388c49c76fe924f7606cfa447fb176007eff5adf41ec81c02e823f7419"}
May 28 01:14:03.705946 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:03.705500    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"
May 28 01:14:03.705946 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:03.705507    2550 scope.go:117] "RemoveContainer" containerID="85ff1a277536d67fc104186a04bf9df23b690c953316789da2e7852b2883aae0"
May 28 01:14:03.713171 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:03.713152    2550 scope.go:117] "RemoveContainer" containerID="2184b0226bcf286978b24ef55ab1ce5b484558459bfbb39770506155f2975c2e"
May 28 01:14:03.720128 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:03.720109    2550 scope.go:117] "RemoveContainer" containerID="42accaa16426049566df16425328a3e81541c57a336b7e8cfde1ffed68631a0c"
May 28 01:14:03.721112 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:03.721093    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"]
May 28 01:14:03.726498 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:03.726475    2550 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-fd659f88c-wn5hv"]
May 28 01:14:03.727450 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:03.727433    2550 scope.go:117] "RemoveContainer" containerID="7704d74b390b9ebbba0d3a66595628f60695c130aa93324364f74e5e79de0374"
May 28 01:14:05.323661 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:05.323632    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72009548-a26c-414b-a747-fdb65a2707f8" path="/var/lib/kubelet/pods/72009548-a26c-414b-a747-fdb65a2707f8/volumes"
May 28 01:14:21.261660 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.261620    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"]
May 28 01:14:21.262091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.261923    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72009548-a26c-414b-a747-fdb65a2707f8" containerName="registry"
May 28 01:14:21.262091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.261935    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="72009548-a26c-414b-a747-fdb65a2707f8" containerName="registry"
May 28 01:14:21.262091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.261946    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72009548-a26c-414b-a747-fdb65a2707f8" containerName="online"
May 28 01:14:21.262091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.261954    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="72009548-a26c-414b-a747-fdb65a2707f8" containerName="online"
May 28 01:14:21.262091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.261962    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bab2d70b-07bc-4f26-a883-f3fc47457f9d" containerName="feast-init"
May 28 01:14:21.262091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.261968    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="bab2d70b-07bc-4f26-a883-f3fc47457f9d" containerName="feast-init"
May 28 01:14:21.262091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.261976    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72009548-a26c-414b-a747-fdb65a2707f8" containerName="feast-init"
May 28 01:14:21.262091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.261981    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="72009548-a26c-414b-a747-fdb65a2707f8" containerName="feast-init"
May 28 01:14:21.262091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.261990    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bab2d70b-07bc-4f26-a883-f3fc47457f9d" containerName="feast-apply"
May 28 01:14:21.262091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.261995    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="bab2d70b-07bc-4f26-a883-f3fc47457f9d" containerName="feast-apply"
May 28 01:14:21.262091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.262003    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fe53fb5-edc8-47fe-9a2c-b1d076959f64" containerName="manager"
May 28 01:14:21.262091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.262008    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fe53fb5-edc8-47fe-9a2c-b1d076959f64" containerName="manager"
May 28 01:14:21.262091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.262014    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="186635cb-ee5f-4051-9b3f-4d2228e1e048" containerName="postgres"
May 28 01:14:21.262091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.262018    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="186635cb-ee5f-4051-9b3f-4d2228e1e048" containerName="postgres"
May 28 01:14:21.262091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.262023    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bab2d70b-07bc-4f26-a883-f3fc47457f9d" containerName="registry"
May 28 01:14:21.262091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.262030    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="bab2d70b-07bc-4f26-a883-f3fc47457f9d" containerName="registry"
May 28 01:14:21.262091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.262040    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="664b6570-2c2e-4260-b2a9-5425eeddbec1" containerName="redis"
May 28 01:14:21.262091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.262047    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="664b6570-2c2e-4260-b2a9-5425eeddbec1" containerName="redis"
May 28 01:14:21.262091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.262055    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72009548-a26c-414b-a747-fdb65a2707f8" containerName="feast-apply"
May 28 01:14:21.262091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.262059    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="72009548-a26c-414b-a747-fdb65a2707f8" containerName="feast-apply"
May 28 01:14:21.262091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.262064    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bab2d70b-07bc-4f26-a883-f3fc47457f9d" containerName="online"
May 28 01:14:21.262091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.262068    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="bab2d70b-07bc-4f26-a883-f3fc47457f9d" containerName="online"
May 28 01:14:21.263149 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.262108    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="664b6570-2c2e-4260-b2a9-5425eeddbec1" containerName="redis"
May 28 01:14:21.263149 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.262119    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="bab2d70b-07bc-4f26-a883-f3fc47457f9d" containerName="online"
May 28 01:14:21.263149 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.262127    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="bab2d70b-07bc-4f26-a883-f3fc47457f9d" containerName="registry"
May 28 01:14:21.263149 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.262134    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="72009548-a26c-414b-a747-fdb65a2707f8" containerName="registry"
May 28 01:14:21.263149 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.262139    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="72009548-a26c-414b-a747-fdb65a2707f8" containerName="online"
May 28 01:14:21.263149 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.262147    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="186635cb-ee5f-4051-9b3f-4d2228e1e048" containerName="postgres"
May 28 01:14:21.263149 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.262154    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="0fe53fb5-edc8-47fe-9a2c-b1d076959f64" containerName="manager"
May 28 01:14:21.267069 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.267049    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:21.269121 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.269083    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 28 01:14:21.269121 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.269083    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 28 01:14:21.269121 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.269091    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-grbtx\""
May 28 01:14:21.269337 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.269095    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 28 01:14:21.269337 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.269158    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 28 01:14:21.269337 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.269096    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 28 01:14:21.269497 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.269470    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 28 01:14:21.278408 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.278382    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"]
May 28 01:14:21.364071 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.364034    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-smdhm\" (UID: \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:21.364071 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.364079    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-smdhm\" (UID: \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:21.364303 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.364109    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-smdhm\" (UID: \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:21.364303 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.364168    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-smdhm\" (UID: \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:21.364303 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.364212    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq5mr\" (UniqueName: \"kubernetes.io/projected/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-kube-api-access-sq5mr\") pod \"feast-simple-feast-setup-664fcfcc58-smdhm\" (UID: \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:21.364303 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.364242    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-smdhm\" (UID: \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:21.465493 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.465455    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-smdhm\" (UID: \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:21.465663 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.465504    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-smdhm\" (UID: \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:21.465663 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.465531    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-smdhm\" (UID: \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:21.465663 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.465553    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-smdhm\" (UID: \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:21.465663 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.465575    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-smdhm\" (UID: \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:21.465663 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.465602    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sq5mr\" (UniqueName: \"kubernetes.io/projected/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-kube-api-access-sq5mr\") pod \"feast-simple-feast-setup-664fcfcc58-smdhm\" (UID: \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:21.465930 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:14:21.465701    2550 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 28 01:14:21.465930 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:14:21.465789    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-ui-tls podName:87d9ef5a-4dd3-4e0b-8fac-32f834b0639b nodeName:}" failed. No retries permitted until 2026-05-28 01:14:21.965767319 +0000 UTC m=+1121.246724390 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-ui-tls") pod "feast-simple-feast-setup-664fcfcc58-smdhm" (UID: "87d9ef5a-4dd3-4e0b-8fac-32f834b0639b") : secret "feast-simple-feast-setup-ui-tls" not found
May 28 01:14:21.466018 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.465983    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-smdhm\" (UID: \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:21.468017 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.467995    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-smdhm\" (UID: \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:21.468229 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.468208    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-smdhm\" (UID: \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:21.468271 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.468213    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-smdhm\" (UID: \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:21.472827 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.472806    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq5mr\" (UniqueName: \"kubernetes.io/projected/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-kube-api-access-sq5mr\") pod \"feast-simple-feast-setup-664fcfcc58-smdhm\" (UID: \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:21.970481 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.970436    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-smdhm\" (UID: \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:21.972983 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:21.972958    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-smdhm\" (UID: \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:22.189019 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:22.188986    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:22.322122 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:22.322087    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"]
May 28 01:14:22.323978 ip-10-0-131-73 kubenswrapper[2550]: W0528 01:14:22.323945    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod87d9ef5a_4dd3_4e0b_8fac_32f834b0639b.slice/crio-ac109be416afee637999c341d83475858e2fae9ecc3ab57720677194a02c9846 WatchSource:0}: Error finding container ac109be416afee637999c341d83475858e2fae9ecc3ab57720677194a02c9846: Status 404 returned error can't find the container with id ac109be416afee637999c341d83475858e2fae9ecc3ab57720677194a02c9846
May 28 01:14:22.325606 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:22.325590    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 01:14:22.758668 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:22.758631    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" event={"ID":"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b","Type":"ContainerStarted","Data":"ac109be416afee637999c341d83475858e2fae9ecc3ab57720677194a02c9846"}
May 28 01:14:37.809428 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:37.809385    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" event={"ID":"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b","Type":"ContainerStarted","Data":"cb25c0faf10e10c15b444ae17281ed8710d21fe409b0a0242fb07d8195eb9862"}
May 28 01:14:42.825173 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:42.825140    2550 generic.go:358] "Generic (PLEG): container finished" podID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerID="cb25c0faf10e10c15b444ae17281ed8710d21fe409b0a0242fb07d8195eb9862" exitCode=0
May 28 01:14:42.825575 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:42.825191    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" event={"ID":"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b","Type":"ContainerDied","Data":"cb25c0faf10e10c15b444ae17281ed8710d21fe409b0a0242fb07d8195eb9862"}
May 28 01:14:43.829674 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:43.829639    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" event={"ID":"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b","Type":"ContainerStarted","Data":"4ff6fd8b0faedce11e8b732743021748a8ecbadc463ddd6485bf7a808c0f53e4"}
May 28 01:14:48.844740 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:48.844698    2550 generic.go:358] "Generic (PLEG): container finished" podID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerID="4ff6fd8b0faedce11e8b732743021748a8ecbadc463ddd6485bf7a808c0f53e4" exitCode=0
May 28 01:14:48.845144 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:48.844757    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" event={"ID":"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b","Type":"ContainerDied","Data":"4ff6fd8b0faedce11e8b732743021748a8ecbadc463ddd6485bf7a808c0f53e4"}
May 28 01:14:49.851856 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:49.851813    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" event={"ID":"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b","Type":"ContainerStarted","Data":"26a976dedfe8971c8437b47c35a9dba7b85bc5dd05ab751647c396d9e15182ea"}
May 28 01:14:49.851856 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:49.851862    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" event={"ID":"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b","Type":"ContainerStarted","Data":"d6883d24770b3135ac150f97c5c446eb247a75eaccf2e29b37429aa2feecc4ec"}
May 28 01:14:49.852398 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:49.851878    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" event={"ID":"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b","Type":"ContainerStarted","Data":"d9c26b43b113c97c90b5db02606eadfc8663cff8a78cc812a47293a10bb872fc"}
May 28 01:14:49.852398 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:49.851892    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" event={"ID":"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b","Type":"ContainerStarted","Data":"2a36b2a85655dbe99a4b106d04d6cbc3649adb4be2bd230a7cf474eeb6848d5f"}
May 28 01:14:49.876772 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:49.876142    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" podStartSLOduration=13.515010814 podStartE2EDuration="28.876121055s" podCreationTimestamp="2026-05-28 01:14:21 +0000 UTC" firstStartedPulling="2026-05-28 01:14:22.325715724 +0000 UTC m=+1121.606672795" lastFinishedPulling="2026-05-28 01:14:37.686825965 +0000 UTC m=+1136.967783036" observedRunningTime="2026-05-28 01:14:49.87327334 +0000 UTC m=+1149.154230451" watchObservedRunningTime="2026-05-28 01:14:49.876121055 +0000 UTC m=+1149.157078149"
May 28 01:14:52.189915 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:52.189869    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:52.189915 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:52.189928    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:52.190490 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:52.189943    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:52.190490 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:52.189957    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:52.190490 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:52.189970    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:52.190490 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:52.189982    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:52.190490 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:52.189994    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:52.190490 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:52.190006    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:52.192158 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:52.192113    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.29:8443: connect: connection refused"
May 28 01:14:52.192158 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:52.192138    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.29:6571: connect: connection refused"
May 28 01:14:52.192362 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:52.192139    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.29:8816: connect: connection refused"
May 28 01:14:52.192362 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:52.192198    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused"
May 28 01:14:55.190557 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:55.190515    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:55.191029 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:55.190982    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:55.191157 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:55.191143    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:55.191691 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:55.191669    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:55.191813 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:55.191731    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:55.192038 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:55.192021    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:55.195147 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:55.195124    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:14:55.875317 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:14:55.875290    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:15:41.287186 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:41.287158    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9z98m_6671c1c1-b674-4a1d-9011-1ca22724b234/ovn-acl-logging/0.log"
May 28 01:15:41.293571 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:41.293549    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9z98m_6671c1c1-b674-4a1d-9011-1ca22724b234/ovn-acl-logging/0.log"
May 28 01:15:57.547730 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:57.547688    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"]
May 28 01:15:57.548201 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:57.548090    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="ui" containerID="cri-o://26a976dedfe8971c8437b47c35a9dba7b85bc5dd05ab751647c396d9e15182ea" gracePeriod=30
May 28 01:15:57.548276 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:57.548204    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="offline" containerID="cri-o://d6883d24770b3135ac150f97c5c446eb247a75eaccf2e29b37429aa2feecc4ec" gracePeriod=30
May 28 01:15:57.548276 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:57.548260    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="online" containerID="cri-o://d9c26b43b113c97c90b5db02606eadfc8663cff8a78cc812a47293a10bb872fc" gracePeriod=30
May 28 01:15:57.548611 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:57.548370    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="registry" containerID="cri-o://2a36b2a85655dbe99a4b106d04d6cbc3649adb4be2bd230a7cf474eeb6848d5f" gracePeriod=30
May 28 01:15:57.821425 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:57.821330    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"]
May 28 01:15:57.825091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:57.825073    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:15:57.827260 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:57.827236    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-vwltf\""
May 28 01:15:57.836310 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:57.836280    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"]
May 28 01:15:57.965583 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:57.965547    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-txsd2\" (UID: \"00171e68-2770-4c85-8b03-3996f824109d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:15:57.965771 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:57.965606    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/00171e68-2770-4c85-8b03-3996f824109d-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-txsd2\" (UID: \"00171e68-2770-4c85-8b03-3996f824109d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:15:57.965771 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:57.965640    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-txsd2\" (UID: \"00171e68-2770-4c85-8b03-3996f824109d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:15:57.965771 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:57.965737    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-txsd2\" (UID: \"00171e68-2770-4c85-8b03-3996f824109d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:15:57.965881 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:57.965775    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-txsd2\" (UID: \"00171e68-2770-4c85-8b03-3996f824109d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:15:57.965881 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:57.965846    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzsqn\" (UniqueName: \"kubernetes.io/projected/00171e68-2770-4c85-8b03-3996f824109d-kube-api-access-qzsqn\") pod \"feast-simple-feast-setup-664fcfcc58-txsd2\" (UID: \"00171e68-2770-4c85-8b03-3996f824109d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:15:58.066846 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:58.066817    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/00171e68-2770-4c85-8b03-3996f824109d-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-txsd2\" (UID: \"00171e68-2770-4c85-8b03-3996f824109d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:15:58.066913 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:58.066885    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-txsd2\" (UID: \"00171e68-2770-4c85-8b03-3996f824109d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:15:58.066959 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:58.066948    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-txsd2\" (UID: \"00171e68-2770-4c85-8b03-3996f824109d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:15:58.067004 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:58.066968    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-txsd2\" (UID: \"00171e68-2770-4c85-8b03-3996f824109d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:15:58.067004 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:58.066997    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qzsqn\" (UniqueName: \"kubernetes.io/projected/00171e68-2770-4c85-8b03-3996f824109d-kube-api-access-qzsqn\") pod \"feast-simple-feast-setup-664fcfcc58-txsd2\" (UID: \"00171e68-2770-4c85-8b03-3996f824109d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:15:58.067091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:58.067024    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-txsd2\" (UID: \"00171e68-2770-4c85-8b03-3996f824109d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:15:58.067145 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:15:58.067085    2550 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found
May 28 01:15:58.067145 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:15:58.067139    2550 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found
May 28 01:15:58.067317 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:15:58.067160    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-registry-tls podName:00171e68-2770-4c85-8b03-3996f824109d nodeName:}" failed. No retries permitted until 2026-05-28 01:15:58.567135948 +0000 UTC m=+1217.848093037 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-registry-tls") pod "feast-simple-feast-setup-664fcfcc58-txsd2" (UID: "00171e68-2770-4c85-8b03-3996f824109d") : secret "feast-simple-feast-setup-registry-tls" not found
May 28 01:15:58.067317 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:58.067166    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/00171e68-2770-4c85-8b03-3996f824109d-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-txsd2\" (UID: \"00171e68-2770-4c85-8b03-3996f824109d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:15:58.067317 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:15:58.067190    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-offline-tls podName:00171e68-2770-4c85-8b03-3996f824109d nodeName:}" failed. No retries permitted until 2026-05-28 01:15:58.567172631 +0000 UTC m=+1217.848129728 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-offline-tls") pod "feast-simple-feast-setup-664fcfcc58-txsd2" (UID: "00171e68-2770-4c85-8b03-3996f824109d") : secret "feast-simple-feast-setup-offline-tls" not found
May 28 01:15:58.069251 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:58.069218    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-txsd2\" (UID: \"00171e68-2770-4c85-8b03-3996f824109d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:15:58.069379 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:58.069362    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-txsd2\" (UID: \"00171e68-2770-4c85-8b03-3996f824109d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:15:58.074973 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:58.074906    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzsqn\" (UniqueName: \"kubernetes.io/projected/00171e68-2770-4c85-8b03-3996f824109d-kube-api-access-qzsqn\") pod \"feast-simple-feast-setup-664fcfcc58-txsd2\" (UID: \"00171e68-2770-4c85-8b03-3996f824109d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:15:58.572549 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:58.572507    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-txsd2\" (UID: \"00171e68-2770-4c85-8b03-3996f824109d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:15:58.572945 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:58.572615    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-txsd2\" (UID: \"00171e68-2770-4c85-8b03-3996f824109d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:15:58.574835 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:58.574818    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-txsd2\" (UID: \"00171e68-2770-4c85-8b03-3996f824109d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:15:58.574896 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:58.574857    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-txsd2\" (UID: \"00171e68-2770-4c85-8b03-3996f824109d\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:15:58.734699 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:58.734659    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:15:58.865589 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:58.865511    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"]
May 28 01:15:58.868627 ip-10-0-131-73 kubenswrapper[2550]: W0528 01:15:58.868600    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00171e68_2770_4c85_8b03_3996f824109d.slice/crio-55834161ff0f9134fe741b82a6d1103ec4c68c1cc50d9e6aadb180086b8e1aba WatchSource:0}: Error finding container 55834161ff0f9134fe741b82a6d1103ec4c68c1cc50d9e6aadb180086b8e1aba: Status 404 returned error can't find the container with id 55834161ff0f9134fe741b82a6d1103ec4c68c1cc50d9e6aadb180086b8e1aba
May 28 01:15:59.054884 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:59.054851    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" event={"ID":"00171e68-2770-4c85-8b03-3996f824109d","Type":"ContainerStarted","Data":"e531fd300441ea57ba57f7552c6a99395a544482d34d4b7fc46a2549c3400a93"}
May 28 01:15:59.055068 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:59.054891    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" event={"ID":"00171e68-2770-4c85-8b03-3996f824109d","Type":"ContainerStarted","Data":"55834161ff0f9134fe741b82a6d1103ec4c68c1cc50d9e6aadb180086b8e1aba"}
May 28 01:15:59.057617 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:59.057589    2550 generic.go:358] "Generic (PLEG): container finished" podID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerID="26a976dedfe8971c8437b47c35a9dba7b85bc5dd05ab751647c396d9e15182ea" exitCode=0
May 28 01:15:59.057617 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:59.057614    2550 generic.go:358] "Generic (PLEG): container finished" podID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerID="d9c26b43b113c97c90b5db02606eadfc8663cff8a78cc812a47293a10bb872fc" exitCode=0
May 28 01:15:59.057767 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:59.057660    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" event={"ID":"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b","Type":"ContainerDied","Data":"26a976dedfe8971c8437b47c35a9dba7b85bc5dd05ab751647c396d9e15182ea"}
May 28 01:15:59.057767 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:15:59.057690    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" event={"ID":"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b","Type":"ContainerDied","Data":"d9c26b43b113c97c90b5db02606eadfc8663cff8a78cc812a47293a10bb872fc"}
May 28 01:16:04.073317 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:04.073283    2550 generic.go:358] "Generic (PLEG): container finished" podID="00171e68-2770-4c85-8b03-3996f824109d" containerID="e531fd300441ea57ba57f7552c6a99395a544482d34d4b7fc46a2549c3400a93" exitCode=0
May 28 01:16:04.073723 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:04.073364    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" event={"ID":"00171e68-2770-4c85-8b03-3996f824109d","Type":"ContainerDied","Data":"e531fd300441ea57ba57f7552c6a99395a544482d34d4b7fc46a2549c3400a93"}
May 28 01:16:05.077288 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:05.077243    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" event={"ID":"00171e68-2770-4c85-8b03-3996f824109d","Type":"ContainerStarted","Data":"04a2d02e89224adc55eec5c9036677a398fda09ee496b6250baf79f0adcecca8"}
May 28 01:16:05.191869 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:05.191824    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.29:8443: connect: connection refused"
May 28 01:16:05.873363 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:05.873319    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused"
May 28 01:16:10.097465 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:10.097428    2550 generic.go:358] "Generic (PLEG): container finished" podID="00171e68-2770-4c85-8b03-3996f824109d" containerID="04a2d02e89224adc55eec5c9036677a398fda09ee496b6250baf79f0adcecca8" exitCode=0
May 28 01:16:10.097895 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:10.097504    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" event={"ID":"00171e68-2770-4c85-8b03-3996f824109d","Type":"ContainerDied","Data":"04a2d02e89224adc55eec5c9036677a398fda09ee496b6250baf79f0adcecca8"}
May 28 01:16:11.104354 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:11.104312    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" event={"ID":"00171e68-2770-4c85-8b03-3996f824109d","Type":"ContainerStarted","Data":"c29b27bfbe9a982f9737438bb947090ef19ce934655e5fd5d148ca851001459c"}
May 28 01:16:11.104862 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:11.104364    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" event={"ID":"00171e68-2770-4c85-8b03-3996f824109d","Type":"ContainerStarted","Data":"98c84b838223011fb82bb5fdb84b4771ddfbd70757486f9ff7f40f9ed51c1983"}
May 28 01:16:11.104862 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:11.104383    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" event={"ID":"00171e68-2770-4c85-8b03-3996f824109d","Type":"ContainerStarted","Data":"18751b2634d75dd73d0a3ff900aa645f1509a1bc3f7abe21408fe63586adbf11"}
May 28 01:16:11.104862 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:11.104395    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" event={"ID":"00171e68-2770-4c85-8b03-3996f824109d","Type":"ContainerStarted","Data":"32bd4d56fe25d671f6db75440980e337e217ed6ea7e8b6b52b142e999d884262"}
May 28 01:16:11.128150 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:11.128091    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" podStartSLOduration=14.128073157 podStartE2EDuration="14.128073157s" podCreationTimestamp="2026-05-28 01:15:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 01:16:11.126991926 +0000 UTC m=+1230.407949044" watchObservedRunningTime="2026-05-28 01:16:11.128073157 +0000 UTC m=+1230.409030253"
May 28 01:16:13.735511 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:13.735471    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:16:13.736215 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:13.736188    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:16:13.736215 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:13.736215    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:16:13.736398 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:13.736229    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:16:13.737465 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:13.737401    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.30:8443: connect: connection refused"
May 28 01:16:13.737602 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:13.737490    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.30:6571: connect: connection refused"
May 28 01:16:13.737602 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:13.737430    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.30:8816: connect: connection refused"
May 28 01:16:13.737602 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:13.737540    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused"
May 28 01:16:15.192435 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:15.192356    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.29:8443: connect: connection refused"
May 28 01:16:15.873296 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:15.873250    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused"
May 28 01:16:16.736177 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:16.736145    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:16:16.736718 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:16.736308    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:16:16.736718 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:16.736606    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:16:16.736718 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:16.736697    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:16:16.736888 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:16.736723    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:16:16.737041 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:16.737022    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:16:16.737182 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:16.737165    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:16:16.740799 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:16.740777    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:16:17.125691 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:17.125613    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:16:17.125691 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:17.125649    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:16:17.128453 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:17.128430    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:16:18.128944 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:18.128917    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:16:25.192217 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:25.192124    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.29:8443: connect: connection refused"
May 28 01:16:25.192683 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:25.192276    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:16:25.873173 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:25.873111    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused"
May 28 01:16:25.873349 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:25.873252    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:16:28.158822 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:28.158790    2550 generic.go:358] "Generic (PLEG): container finished" podID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerID="d6883d24770b3135ac150f97c5c446eb247a75eaccf2e29b37429aa2feecc4ec" exitCode=137
May 28 01:16:28.158822 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:28.158816    2550 generic.go:358] "Generic (PLEG): container finished" podID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerID="2a36b2a85655dbe99a4b106d04d6cbc3649adb4be2bd230a7cf474eeb6848d5f" exitCode=137
May 28 01:16:28.159210 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:28.158843    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" event={"ID":"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b","Type":"ContainerDied","Data":"d6883d24770b3135ac150f97c5c446eb247a75eaccf2e29b37429aa2feecc4ec"}
May 28 01:16:28.159210 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:28.158870    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" event={"ID":"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b","Type":"ContainerDied","Data":"2a36b2a85655dbe99a4b106d04d6cbc3649adb4be2bd230a7cf474eeb6848d5f"}
May 28 01:16:28.188980 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:28.188958    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:16:28.221879 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:28.221849    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-registry-tls\") pod \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\" (UID: \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\") "
May 28 01:16:28.222035 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:28.221903    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-online-tls\") pod \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\" (UID: \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\") "
May 28 01:16:28.222035 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:28.221933    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-offline-tls\") pod \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\" (UID: \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\") "
May 28 01:16:28.222035 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:28.221969    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-ui-tls\") pod \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\" (UID: \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\") "
May 28 01:16:28.222035 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:28.222034    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sq5mr\" (UniqueName: \"kubernetes.io/projected/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-kube-api-access-sq5mr\") pod \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\" (UID: \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\") "
May 28 01:16:28.222259 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:28.222070    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-feast-data\") pod \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\" (UID: \"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b\") "
May 28 01:16:28.222828 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:28.222794    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-feast-data" (OuterVolumeSpecName: "feast-data") pod "87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" (UID: "87d9ef5a-4dd3-4e0b-8fac-32f834b0639b"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 01:16:28.224213 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:28.224180    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" (UID: "87d9ef5a-4dd3-4e0b-8fac-32f834b0639b"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:16:28.224213 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:28.224195    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-online-tls" (OuterVolumeSpecName: "online-tls") pod "87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" (UID: "87d9ef5a-4dd3-4e0b-8fac-32f834b0639b"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:16:28.224371 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:28.224297    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" (UID: "87d9ef5a-4dd3-4e0b-8fac-32f834b0639b"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:16:28.224576 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:28.224560    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" (UID: "87d9ef5a-4dd3-4e0b-8fac-32f834b0639b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:16:28.224576 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:28.224564    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-kube-api-access-sq5mr" (OuterVolumeSpecName: "kube-api-access-sq5mr") pod "87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" (UID: "87d9ef5a-4dd3-4e0b-8fac-32f834b0639b"). InnerVolumeSpecName "kube-api-access-sq5mr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 01:16:28.323196 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:28.323109    2550 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-registry-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:16:28.323196 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:28.323136    2550 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-online-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:16:28.323196 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:28.323148    2550 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-offline-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:16:28.323196 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:28.323158    2550 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-ui-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:16:28.323196 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:28.323169    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sq5mr\" (UniqueName: \"kubernetes.io/projected/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-kube-api-access-sq5mr\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:16:28.323196 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:28.323181    2550 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b-feast-data\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:16:29.164872 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:29.164839    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm" event={"ID":"87d9ef5a-4dd3-4e0b-8fac-32f834b0639b","Type":"ContainerDied","Data":"ac109be416afee637999c341d83475858e2fae9ecc3ab57720677194a02c9846"}
May 28 01:16:29.165327 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:29.164887    2550 scope.go:117] "RemoveContainer" containerID="26a976dedfe8971c8437b47c35a9dba7b85bc5dd05ab751647c396d9e15182ea"
May 28 01:16:29.165327 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:29.164948    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"
May 28 01:16:29.174963 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:29.174943    2550 scope.go:117] "RemoveContainer" containerID="d6883d24770b3135ac150f97c5c446eb247a75eaccf2e29b37429aa2feecc4ec"
May 28 01:16:29.181794 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:29.181777    2550 scope.go:117] "RemoveContainer" containerID="d9c26b43b113c97c90b5db02606eadfc8663cff8a78cc812a47293a10bb872fc"
May 28 01:16:29.188424 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:29.188390    2550 scope.go:117] "RemoveContainer" containerID="2a36b2a85655dbe99a4b106d04d6cbc3649adb4be2bd230a7cf474eeb6848d5f"
May 28 01:16:29.194318 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:29.194292    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"]
May 28 01:16:29.195938 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:29.195921    2550 scope.go:117] "RemoveContainer" containerID="4ff6fd8b0faedce11e8b732743021748a8ecbadc463ddd6485bf7a808c0f53e4"
May 28 01:16:29.202360 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:29.202337    2550 scope.go:117] "RemoveContainer" containerID="cb25c0faf10e10c15b444ae17281ed8710d21fe409b0a0242fb07d8195eb9862"
May 28 01:16:29.208761 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:29.208739    2550 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-smdhm"]
May 28 01:16:29.323278 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:16:29.323247    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" path="/var/lib/kubelet/pods/87d9ef5a-4dd3-4e0b-8fac-32f834b0639b/volumes"
May 28 01:17:19.678751 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.678713    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"]
May 28 01:17:19.679397 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.679109    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="registry"
May 28 01:17:19.679397 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.679127    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="registry"
May 28 01:17:19.679397 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.679141    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="ui"
May 28 01:17:19.679397 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.679149    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="ui"
May 28 01:17:19.679397 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.679159    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="feast-apply"
May 28 01:17:19.679397 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.679167    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="feast-apply"
May 28 01:17:19.679397 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.679186    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="offline"
May 28 01:17:19.679397 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.679194    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="offline"
May 28 01:17:19.679397 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.679206    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="feast-init"
May 28 01:17:19.679397 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.679216    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="feast-init"
May 28 01:17:19.679397 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.679230    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="online"
May 28 01:17:19.679397 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.679238    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="online"
May 28 01:17:19.679397 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.679303    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="online"
May 28 01:17:19.679397 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.679314    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="offline"
May 28 01:17:19.679397 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.679324    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="ui"
May 28 01:17:19.679397 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.679335    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="87d9ef5a-4dd3-4e0b-8fac-32f834b0639b" containerName="registry"
May 28 01:17:19.682869 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.682847    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:19.685281 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.685255    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 28 01:17:19.685432 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.685250    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 28 01:17:19.685513 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.685426    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 28 01:17:19.685592 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.685575    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-w8tbc\""
May 28 01:17:19.685637 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.685610    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 28 01:17:19.685690 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.685671    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 28 01:17:19.685975 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.685958    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 28 01:17:19.690894 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.690871    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"]
May 28 01:17:19.837134 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.837082    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4d05ee48-ebbe-49ff-8632-1125083a8075-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-wzx76\" (UID: \"4d05ee48-ebbe-49ff-8632-1125083a8075\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:19.837323 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.837155    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx8h6\" (UniqueName: \"kubernetes.io/projected/4d05ee48-ebbe-49ff-8632-1125083a8075-kube-api-access-bx8h6\") pod \"feast-simple-feast-remote-setup-5cff94746b-wzx76\" (UID: \"4d05ee48-ebbe-49ff-8632-1125083a8075\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:19.837323 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.837206    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4d05ee48-ebbe-49ff-8632-1125083a8075-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-wzx76\" (UID: \"4d05ee48-ebbe-49ff-8632-1125083a8075\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:19.837323 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.837230    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4d05ee48-ebbe-49ff-8632-1125083a8075-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-wzx76\" (UID: \"4d05ee48-ebbe-49ff-8632-1125083a8075\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:19.837323 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.837254    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4d05ee48-ebbe-49ff-8632-1125083a8075-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-wzx76\" (UID: \"4d05ee48-ebbe-49ff-8632-1125083a8075\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:19.837323 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.837280    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/4d05ee48-ebbe-49ff-8632-1125083a8075-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-wzx76\" (UID: \"4d05ee48-ebbe-49ff-8632-1125083a8075\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:19.938309 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.938202    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bx8h6\" (UniqueName: \"kubernetes.io/projected/4d05ee48-ebbe-49ff-8632-1125083a8075-kube-api-access-bx8h6\") pod \"feast-simple-feast-remote-setup-5cff94746b-wzx76\" (UID: \"4d05ee48-ebbe-49ff-8632-1125083a8075\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:19.938309 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.938269    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4d05ee48-ebbe-49ff-8632-1125083a8075-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-wzx76\" (UID: \"4d05ee48-ebbe-49ff-8632-1125083a8075\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:19.938309 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.938301    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4d05ee48-ebbe-49ff-8632-1125083a8075-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-wzx76\" (UID: \"4d05ee48-ebbe-49ff-8632-1125083a8075\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:19.938660 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.938323    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4d05ee48-ebbe-49ff-8632-1125083a8075-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-wzx76\" (UID: \"4d05ee48-ebbe-49ff-8632-1125083a8075\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:19.938660 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.938352    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/4d05ee48-ebbe-49ff-8632-1125083a8075-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-wzx76\" (UID: \"4d05ee48-ebbe-49ff-8632-1125083a8075\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:19.938660 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.938410    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4d05ee48-ebbe-49ff-8632-1125083a8075-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-wzx76\" (UID: \"4d05ee48-ebbe-49ff-8632-1125083a8075\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:19.938660 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:17:19.938484    2550 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found
May 28 01:17:19.938660 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:17:19.938565    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d05ee48-ebbe-49ff-8632-1125083a8075-ui-tls podName:4d05ee48-ebbe-49ff-8632-1125083a8075 nodeName:}" failed. No retries permitted until 2026-05-28 01:17:20.438543258 +0000 UTC m=+1299.719500351 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/4d05ee48-ebbe-49ff-8632-1125083a8075-ui-tls") pod "feast-simple-feast-remote-setup-5cff94746b-wzx76" (UID: "4d05ee48-ebbe-49ff-8632-1125083a8075") : secret "feast-simple-feast-remote-setup-ui-tls" not found
May 28 01:17:19.938901 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.938880    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4d05ee48-ebbe-49ff-8632-1125083a8075-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-wzx76\" (UID: \"4d05ee48-ebbe-49ff-8632-1125083a8075\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:19.939116 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.939097    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/4d05ee48-ebbe-49ff-8632-1125083a8075-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-wzx76\" (UID: \"4d05ee48-ebbe-49ff-8632-1125083a8075\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:19.940908 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.940887    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4d05ee48-ebbe-49ff-8632-1125083a8075-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-wzx76\" (UID: \"4d05ee48-ebbe-49ff-8632-1125083a8075\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:19.940992 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.940978    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4d05ee48-ebbe-49ff-8632-1125083a8075-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-wzx76\" (UID: \"4d05ee48-ebbe-49ff-8632-1125083a8075\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:19.945958 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:19.945936    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx8h6\" (UniqueName: \"kubernetes.io/projected/4d05ee48-ebbe-49ff-8632-1125083a8075-kube-api-access-bx8h6\") pod \"feast-simple-feast-remote-setup-5cff94746b-wzx76\" (UID: \"4d05ee48-ebbe-49ff-8632-1125083a8075\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:20.443155 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:20.443124    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4d05ee48-ebbe-49ff-8632-1125083a8075-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-wzx76\" (UID: \"4d05ee48-ebbe-49ff-8632-1125083a8075\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:20.445549 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:20.445518    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4d05ee48-ebbe-49ff-8632-1125083a8075-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-wzx76\" (UID: \"4d05ee48-ebbe-49ff-8632-1125083a8075\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:20.592598 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:20.592561    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:20.726634 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:20.726545    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"]
May 28 01:17:20.729984 ip-10-0-131-73 kubenswrapper[2550]: W0528 01:17:20.729958    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d05ee48_ebbe_49ff_8632_1125083a8075.slice/crio-1f64b21a14145418a767fd631135cf5853f33c954fba37b6302dd1f29708d056 WatchSource:0}: Error finding container 1f64b21a14145418a767fd631135cf5853f33c954fba37b6302dd1f29708d056: Status 404 returned error can't find the container with id 1f64b21a14145418a767fd631135cf5853f33c954fba37b6302dd1f29708d056
May 28 01:17:21.316000 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:21.315960    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76" event={"ID":"4d05ee48-ebbe-49ff-8632-1125083a8075","Type":"ContainerStarted","Data":"27cd38013d385c07f99dfa5f62c24eb10f3efc9f57728e81326f3743c98f9f24"}
May 28 01:17:21.316000 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:21.315999    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76" event={"ID":"4d05ee48-ebbe-49ff-8632-1125083a8075","Type":"ContainerStarted","Data":"1f64b21a14145418a767fd631135cf5853f33c954fba37b6302dd1f29708d056"}
May 28 01:17:26.334178 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:26.334143    2550 generic.go:358] "Generic (PLEG): container finished" podID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerID="27cd38013d385c07f99dfa5f62c24eb10f3efc9f57728e81326f3743c98f9f24" exitCode=0
May 28 01:17:26.334578 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:26.334220    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76" event={"ID":"4d05ee48-ebbe-49ff-8632-1125083a8075","Type":"ContainerDied","Data":"27cd38013d385c07f99dfa5f62c24eb10f3efc9f57728e81326f3743c98f9f24"}
May 28 01:17:27.338251 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:27.338215    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76" event={"ID":"4d05ee48-ebbe-49ff-8632-1125083a8075","Type":"ContainerStarted","Data":"2a4f0ab42f03f3d17f73319be84425e35a24858fc2e261f1c550690052c3412c"}
May 28 01:17:32.353899 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:32.353803    2550 generic.go:358] "Generic (PLEG): container finished" podID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerID="2a4f0ab42f03f3d17f73319be84425e35a24858fc2e261f1c550690052c3412c" exitCode=0
May 28 01:17:32.353899 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:32.353864    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76" event={"ID":"4d05ee48-ebbe-49ff-8632-1125083a8075","Type":"ContainerDied","Data":"2a4f0ab42f03f3d17f73319be84425e35a24858fc2e261f1c550690052c3412c"}
May 28 01:17:33.359807 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:33.359774    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76" event={"ID":"4d05ee48-ebbe-49ff-8632-1125083a8075","Type":"ContainerStarted","Data":"ac4af642d735283aecaadeeaa32becff08be6b765ae521e768cb75758fa45b90"}
May 28 01:17:33.359807 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:33.359813    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76" event={"ID":"4d05ee48-ebbe-49ff-8632-1125083a8075","Type":"ContainerStarted","Data":"58d9ed10be64f2137f90c2292cb4cd1b84ec993b4dc06748976dd2cd5b2ad78f"}
May 28 01:17:33.360387 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:33.359822    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76" event={"ID":"4d05ee48-ebbe-49ff-8632-1125083a8075","Type":"ContainerStarted","Data":"ffe5bed9ead47ec184a52e880f56670c334f94585c9c5e1881dcb26ea8a53425"}
May 28 01:17:33.382348 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:33.382287    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76" podStartSLOduration=14.382270588 podStartE2EDuration="14.382270588s" podCreationTimestamp="2026-05-28 01:17:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 01:17:33.381370903 +0000 UTC m=+1312.662328000" watchObservedRunningTime="2026-05-28 01:17:33.382270588 +0000 UTC m=+1312.663227680"
May 28 01:17:35.593352 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:35.593313    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:35.593911 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:35.593877    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:35.593911 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:35.593904    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:35.595182 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:35.595145    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76" podUID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.31:8443: connect: connection refused"
May 28 01:17:35.595279 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:35.595197    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76" podUID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerName="online" probeResult="failure" output="Get \"https://10.133.0.31:6567/health\": dial tcp 10.133.0.31:6567: connect: connection refused"
May 28 01:17:35.595279 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:35.595161    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76" podUID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.31:8816: connect: connection refused"
May 28 01:17:38.594243 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:38.594214    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:38.594646 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:38.594334    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:38.594806 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:38.594788    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:38.594896 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:38.594821    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:38.595011 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:38.594994    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:38.595229 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:38.595211    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:38.598325 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:38.598307    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:39.380080 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:39.380052    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:17:39.383090 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:17:39.383069    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:18:40.857057 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:18:40.857021    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"]
May 28 01:18:40.857719 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:18:40.857453    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76" podUID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerName="online" containerID="cri-o://ffe5bed9ead47ec184a52e880f56670c334f94585c9c5e1881dcb26ea8a53425" gracePeriod=30
May 28 01:18:40.857719 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:18:40.857520    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76" podUID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerName="ui" containerID="cri-o://ac4af642d735283aecaadeeaa32becff08be6b765ae521e768cb75758fa45b90" gracePeriod=30
May 28 01:18:40.857719 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:18:40.857554    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76" podUID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerName="offline" containerID="cri-o://58d9ed10be64f2137f90c2292cb4cd1b84ec993b4dc06748976dd2cd5b2ad78f" gracePeriod=30
May 28 01:18:40.991681 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:18:40.991643    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"]
May 28 01:18:40.992247 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:18:40.992005    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="registry" containerID="cri-o://32bd4d56fe25d671f6db75440980e337e217ed6ea7e8b6b52b142e999d884262" gracePeriod=30
May 28 01:18:40.992247 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:18:40.992135    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="online" containerID="cri-o://18751b2634d75dd73d0a3ff900aa645f1509a1bc3f7abe21408fe63586adbf11" gracePeriod=30
May 28 01:18:40.992247 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:18:40.992122    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="offline" containerID="cri-o://98c84b838223011fb82bb5fdb84b4771ddfbd70757486f9ff7f40f9ed51c1983" gracePeriod=30
May 28 01:18:40.993535 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:18:40.992578    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="ui" containerID="cri-o://c29b27bfbe9a982f9737438bb947090ef19ce934655e5fd5d148ca851001459c" gracePeriod=30
May 28 01:18:41.564844 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:18:41.564811    2550 generic.go:358] "Generic (PLEG): container finished" podID="00171e68-2770-4c85-8b03-3996f824109d" containerID="c29b27bfbe9a982f9737438bb947090ef19ce934655e5fd5d148ca851001459c" exitCode=0
May 28 01:18:41.565022 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:18:41.564886    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" event={"ID":"00171e68-2770-4c85-8b03-3996f824109d","Type":"ContainerDied","Data":"c29b27bfbe9a982f9737438bb947090ef19ce934655e5fd5d148ca851001459c"}
May 28 01:18:41.567403 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:18:41.567376    2550 generic.go:358] "Generic (PLEG): container finished" podID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerID="ac4af642d735283aecaadeeaa32becff08be6b765ae521e768cb75758fa45b90" exitCode=0
May 28 01:18:41.567561 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:18:41.567436    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76" event={"ID":"4d05ee48-ebbe-49ff-8632-1125083a8075","Type":"ContainerDied","Data":"ac4af642d735283aecaadeeaa32becff08be6b765ae521e768cb75758fa45b90"}
May 28 01:18:42.572260 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:18:42.572222    2550 generic.go:358] "Generic (PLEG): container finished" podID="00171e68-2770-4c85-8b03-3996f824109d" containerID="18751b2634d75dd73d0a3ff900aa645f1509a1bc3f7abe21408fe63586adbf11" exitCode=0
May 28 01:18:42.572715 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:18:42.572295    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" event={"ID":"00171e68-2770-4c85-8b03-3996f824109d","Type":"ContainerDied","Data":"18751b2634d75dd73d0a3ff900aa645f1509a1bc3f7abe21408fe63586adbf11"}
May 28 01:18:42.574439 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:18:42.574395    2550 generic.go:358] "Generic (PLEG): container finished" podID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerID="ffe5bed9ead47ec184a52e880f56670c334f94585c9c5e1881dcb26ea8a53425" exitCode=0
May 28 01:18:42.574554 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:18:42.574442    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76" event={"ID":"4d05ee48-ebbe-49ff-8632-1125083a8075","Type":"ContainerDied","Data":"ffe5bed9ead47ec184a52e880f56670c334f94585c9c5e1881dcb26ea8a53425"}
May 28 01:18:46.737269 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:18:46.737221    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.30:8443: connect: connection refused"
May 28 01:18:47.126480 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:18:47.126376    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused"
May 28 01:18:48.595014 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:18:48.594973    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76" podUID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.31:8443: connect: connection refused"
May 28 01:18:49.380853 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:18:49.380804    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76" podUID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerName="online" probeResult="failure" output="Get \"https://10.133.0.31:6567/health\": dial tcp 10.133.0.31:6567: connect: connection refused"
May 28 01:18:56.736901 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:18:56.736852    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.30:8443: connect: connection refused"
May 28 01:18:57.126148 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:18:57.126051    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused"
May 28 01:18:58.595194 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:18:58.595143    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76" podUID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.31:8443: connect: connection refused"
May 28 01:18:59.381150 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:18:59.381112    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76" podUID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerName="online" probeResult="failure" output="Get \"https://10.133.0.31:6567/health\": dial tcp 10.133.0.31:6567: connect: connection refused"
May 28 01:19:06.737278 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:06.737232    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.30:8443: connect: connection refused"
May 28 01:19:06.737701 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:06.737361    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:19:07.126798 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:07.126708    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused"
May 28 01:19:07.126957 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:07.126822    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:19:08.595353 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:08.595305    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76" podUID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.31:8443: connect: connection refused"
May 28 01:19:08.595813 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:08.595485    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:19:09.381113 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:09.381072    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76" podUID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerName="online" probeResult="failure" output="Get \"https://10.133.0.31:6567/health\": dial tcp 10.133.0.31:6567: connect: connection refused"
May 28 01:19:09.381315 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:09.381199    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:19:11.095250 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.095228    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:19:11.188952 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.188920    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4d05ee48-ebbe-49ff-8632-1125083a8075-offline-tls\") pod \"4d05ee48-ebbe-49ff-8632-1125083a8075\" (UID: \"4d05ee48-ebbe-49ff-8632-1125083a8075\") "
May 28 01:19:11.189114 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.188973    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4d05ee48-ebbe-49ff-8632-1125083a8075-feast-data\") pod \"4d05ee48-ebbe-49ff-8632-1125083a8075\" (UID: \"4d05ee48-ebbe-49ff-8632-1125083a8075\") "
May 28 01:19:11.189114 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.188993    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4d05ee48-ebbe-49ff-8632-1125083a8075-online-tls\") pod \"4d05ee48-ebbe-49ff-8632-1125083a8075\" (UID: \"4d05ee48-ebbe-49ff-8632-1125083a8075\") "
May 28 01:19:11.189114 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.189022    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bx8h6\" (UniqueName: \"kubernetes.io/projected/4d05ee48-ebbe-49ff-8632-1125083a8075-kube-api-access-bx8h6\") pod \"4d05ee48-ebbe-49ff-8632-1125083a8075\" (UID: \"4d05ee48-ebbe-49ff-8632-1125083a8075\") "
May 28 01:19:11.189114 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.189054    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/4d05ee48-ebbe-49ff-8632-1125083a8075-registry-tls\") pod \"4d05ee48-ebbe-49ff-8632-1125083a8075\" (UID: \"4d05ee48-ebbe-49ff-8632-1125083a8075\") "
May 28 01:19:11.189114 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.189080    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4d05ee48-ebbe-49ff-8632-1125083a8075-ui-tls\") pod \"4d05ee48-ebbe-49ff-8632-1125083a8075\" (UID: \"4d05ee48-ebbe-49ff-8632-1125083a8075\") "
May 28 01:19:11.189528 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.189501    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d05ee48-ebbe-49ff-8632-1125083a8075-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4d05ee48-ebbe-49ff-8632-1125083a8075" (UID: "4d05ee48-ebbe-49ff-8632-1125083a8075"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 01:19:11.189621 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.189569    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d05ee48-ebbe-49ff-8632-1125083a8075-feast-data" (OuterVolumeSpecName: "feast-data") pod "4d05ee48-ebbe-49ff-8632-1125083a8075" (UID: "4d05ee48-ebbe-49ff-8632-1125083a8075"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 01:19:11.191260 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.191232    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d05ee48-ebbe-49ff-8632-1125083a8075-online-tls" (OuterVolumeSpecName: "online-tls") pod "4d05ee48-ebbe-49ff-8632-1125083a8075" (UID: "4d05ee48-ebbe-49ff-8632-1125083a8075"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:19:11.191359 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.191258    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d05ee48-ebbe-49ff-8632-1125083a8075-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "4d05ee48-ebbe-49ff-8632-1125083a8075" (UID: "4d05ee48-ebbe-49ff-8632-1125083a8075"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:19:11.191581 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.191558    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d05ee48-ebbe-49ff-8632-1125083a8075-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "4d05ee48-ebbe-49ff-8632-1125083a8075" (UID: "4d05ee48-ebbe-49ff-8632-1125083a8075"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:19:11.191581 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.191570    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d05ee48-ebbe-49ff-8632-1125083a8075-kube-api-access-bx8h6" (OuterVolumeSpecName: "kube-api-access-bx8h6") pod "4d05ee48-ebbe-49ff-8632-1125083a8075" (UID: "4d05ee48-ebbe-49ff-8632-1125083a8075"). InnerVolumeSpecName "kube-api-access-bx8h6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 01:19:11.290144 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.290098    2550 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4d05ee48-ebbe-49ff-8632-1125083a8075-offline-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:19:11.290144 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.290134    2550 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4d05ee48-ebbe-49ff-8632-1125083a8075-feast-data\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:19:11.290144 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.290146    2550 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4d05ee48-ebbe-49ff-8632-1125083a8075-online-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:19:11.290144 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.290155    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bx8h6\" (UniqueName: \"kubernetes.io/projected/4d05ee48-ebbe-49ff-8632-1125083a8075-kube-api-access-bx8h6\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:19:11.290446 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.290166    2550 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/4d05ee48-ebbe-49ff-8632-1125083a8075-registry-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:19:11.290446 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.290174    2550 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4d05ee48-ebbe-49ff-8632-1125083a8075-ui-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:19:11.631599 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.631574    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:19:11.658430 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.658385    2550 generic.go:358] "Generic (PLEG): container finished" podID="00171e68-2770-4c85-8b03-3996f824109d" containerID="98c84b838223011fb82bb5fdb84b4771ddfbd70757486f9ff7f40f9ed51c1983" exitCode=137
May 28 01:19:11.658430 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.658432    2550 generic.go:358] "Generic (PLEG): container finished" podID="00171e68-2770-4c85-8b03-3996f824109d" containerID="32bd4d56fe25d671f6db75440980e337e217ed6ea7e8b6b52b142e999d884262" exitCode=137
May 28 01:19:11.658661 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.658493    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" event={"ID":"00171e68-2770-4c85-8b03-3996f824109d","Type":"ContainerDied","Data":"98c84b838223011fb82bb5fdb84b4771ddfbd70757486f9ff7f40f9ed51c1983"}
May 28 01:19:11.658661 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.658513    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"
May 28 01:19:11.658661 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.658521    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" event={"ID":"00171e68-2770-4c85-8b03-3996f824109d","Type":"ContainerDied","Data":"32bd4d56fe25d671f6db75440980e337e217ed6ea7e8b6b52b142e999d884262"}
May 28 01:19:11.658661 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.658535    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2" event={"ID":"00171e68-2770-4c85-8b03-3996f824109d","Type":"ContainerDied","Data":"55834161ff0f9134fe741b82a6d1103ec4c68c1cc50d9e6aadb180086b8e1aba"}
May 28 01:19:11.658661 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.658553    2550 scope.go:117] "RemoveContainer" containerID="c29b27bfbe9a982f9737438bb947090ef19ce934655e5fd5d148ca851001459c"
May 28 01:19:11.661247 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.661215    2550 generic.go:358] "Generic (PLEG): container finished" podID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerID="58d9ed10be64f2137f90c2292cb4cd1b84ec993b4dc06748976dd2cd5b2ad78f" exitCode=137
May 28 01:19:11.661398 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.661299    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"
May 28 01:19:11.661398 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.661313    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76" event={"ID":"4d05ee48-ebbe-49ff-8632-1125083a8075","Type":"ContainerDied","Data":"58d9ed10be64f2137f90c2292cb4cd1b84ec993b4dc06748976dd2cd5b2ad78f"}
May 28 01:19:11.661398 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.661352    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76" event={"ID":"4d05ee48-ebbe-49ff-8632-1125083a8075","Type":"ContainerDied","Data":"1f64b21a14145418a767fd631135cf5853f33c954fba37b6302dd1f29708d056"}
May 28 01:19:11.668587 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.668548    2550 scope.go:117] "RemoveContainer" containerID="98c84b838223011fb82bb5fdb84b4771ddfbd70757486f9ff7f40f9ed51c1983"
May 28 01:19:11.675951 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.675927    2550 scope.go:117] "RemoveContainer" containerID="18751b2634d75dd73d0a3ff900aa645f1509a1bc3f7abe21408fe63586adbf11"
May 28 01:19:11.678858 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.678835    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"]
May 28 01:19:11.681871 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.681848    2550 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-wzx76"]
May 28 01:19:11.683967 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.683949    2550 scope.go:117] "RemoveContainer" containerID="32bd4d56fe25d671f6db75440980e337e217ed6ea7e8b6b52b142e999d884262"
May 28 01:19:11.690940 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.690922    2550 scope.go:117] "RemoveContainer" containerID="04a2d02e89224adc55eec5c9036677a398fda09ee496b6250baf79f0adcecca8"
May 28 01:19:11.694007 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.693989    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-ui-tls\") pod \"00171e68-2770-4c85-8b03-3996f824109d\" (UID: \"00171e68-2770-4c85-8b03-3996f824109d\") "
May 28 01:19:11.694086 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.694022    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-online-tls\") pod \"00171e68-2770-4c85-8b03-3996f824109d\" (UID: \"00171e68-2770-4c85-8b03-3996f824109d\") "
May 28 01:19:11.694086 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.694061    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-registry-tls\") pod \"00171e68-2770-4c85-8b03-3996f824109d\" (UID: \"00171e68-2770-4c85-8b03-3996f824109d\") "
May 28 01:19:11.694198 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.694106    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzsqn\" (UniqueName: \"kubernetes.io/projected/00171e68-2770-4c85-8b03-3996f824109d-kube-api-access-qzsqn\") pod \"00171e68-2770-4c85-8b03-3996f824109d\" (UID: \"00171e68-2770-4c85-8b03-3996f824109d\") "
May 28 01:19:11.694198 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.694147    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-offline-tls\") pod \"00171e68-2770-4c85-8b03-3996f824109d\" (UID: \"00171e68-2770-4c85-8b03-3996f824109d\") "
May 28 01:19:11.694198 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.694170    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/00171e68-2770-4c85-8b03-3996f824109d-feast-data\") pod \"00171e68-2770-4c85-8b03-3996f824109d\" (UID: \"00171e68-2770-4c85-8b03-3996f824109d\") "
May 28 01:19:11.694882 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.694853    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00171e68-2770-4c85-8b03-3996f824109d-feast-data" (OuterVolumeSpecName: "feast-data") pod "00171e68-2770-4c85-8b03-3996f824109d" (UID: "00171e68-2770-4c85-8b03-3996f824109d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 01:19:11.696349 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.696316    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-online-tls" (OuterVolumeSpecName: "online-tls") pod "00171e68-2770-4c85-8b03-3996f824109d" (UID: "00171e68-2770-4c85-8b03-3996f824109d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:19:11.696349 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.696337    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "00171e68-2770-4c85-8b03-3996f824109d" (UID: "00171e68-2770-4c85-8b03-3996f824109d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:19:11.696545 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.696407    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "00171e68-2770-4c85-8b03-3996f824109d" (UID: "00171e68-2770-4c85-8b03-3996f824109d"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:19:11.696659 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.696635    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "00171e68-2770-4c85-8b03-3996f824109d" (UID: "00171e68-2770-4c85-8b03-3996f824109d"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:19:11.696659 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.696647    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00171e68-2770-4c85-8b03-3996f824109d-kube-api-access-qzsqn" (OuterVolumeSpecName: "kube-api-access-qzsqn") pod "00171e68-2770-4c85-8b03-3996f824109d" (UID: "00171e68-2770-4c85-8b03-3996f824109d"). InnerVolumeSpecName "kube-api-access-qzsqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 01:19:11.698611 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.698591    2550 scope.go:117] "RemoveContainer" containerID="e531fd300441ea57ba57f7552c6a99395a544482d34d4b7fc46a2549c3400a93"
May 28 01:19:11.705267 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.705243    2550 scope.go:117] "RemoveContainer" containerID="c29b27bfbe9a982f9737438bb947090ef19ce934655e5fd5d148ca851001459c"
May 28 01:19:11.705549 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:19:11.705523    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c29b27bfbe9a982f9737438bb947090ef19ce934655e5fd5d148ca851001459c\": container with ID starting with c29b27bfbe9a982f9737438bb947090ef19ce934655e5fd5d148ca851001459c not found: ID does not exist" containerID="c29b27bfbe9a982f9737438bb947090ef19ce934655e5fd5d148ca851001459c"
May 28 01:19:11.705626 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.705556    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c29b27bfbe9a982f9737438bb947090ef19ce934655e5fd5d148ca851001459c"} err="failed to get container status \"c29b27bfbe9a982f9737438bb947090ef19ce934655e5fd5d148ca851001459c\": rpc error: code = NotFound desc = could not find container \"c29b27bfbe9a982f9737438bb947090ef19ce934655e5fd5d148ca851001459c\": container with ID starting with c29b27bfbe9a982f9737438bb947090ef19ce934655e5fd5d148ca851001459c not found: ID does not exist"
May 28 01:19:11.705626 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.705578    2550 scope.go:117] "RemoveContainer" containerID="98c84b838223011fb82bb5fdb84b4771ddfbd70757486f9ff7f40f9ed51c1983"
May 28 01:19:11.705812 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:19:11.705797    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98c84b838223011fb82bb5fdb84b4771ddfbd70757486f9ff7f40f9ed51c1983\": container with ID starting with 98c84b838223011fb82bb5fdb84b4771ddfbd70757486f9ff7f40f9ed51c1983 not found: ID does not exist" containerID="98c84b838223011fb82bb5fdb84b4771ddfbd70757486f9ff7f40f9ed51c1983"
May 28 01:19:11.705853 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.705816    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98c84b838223011fb82bb5fdb84b4771ddfbd70757486f9ff7f40f9ed51c1983"} err="failed to get container status \"98c84b838223011fb82bb5fdb84b4771ddfbd70757486f9ff7f40f9ed51c1983\": rpc error: code = NotFound desc = could not find container \"98c84b838223011fb82bb5fdb84b4771ddfbd70757486f9ff7f40f9ed51c1983\": container with ID starting with 98c84b838223011fb82bb5fdb84b4771ddfbd70757486f9ff7f40f9ed51c1983 not found: ID does not exist"
May 28 01:19:11.705853 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.705829    2550 scope.go:117] "RemoveContainer" containerID="18751b2634d75dd73d0a3ff900aa645f1509a1bc3f7abe21408fe63586adbf11"
May 28 01:19:11.706086 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:19:11.706065    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18751b2634d75dd73d0a3ff900aa645f1509a1bc3f7abe21408fe63586adbf11\": container with ID starting with 18751b2634d75dd73d0a3ff900aa645f1509a1bc3f7abe21408fe63586adbf11 not found: ID does not exist" containerID="18751b2634d75dd73d0a3ff900aa645f1509a1bc3f7abe21408fe63586adbf11"
May 28 01:19:11.706172 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.706089    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18751b2634d75dd73d0a3ff900aa645f1509a1bc3f7abe21408fe63586adbf11"} err="failed to get container status \"18751b2634d75dd73d0a3ff900aa645f1509a1bc3f7abe21408fe63586adbf11\": rpc error: code = NotFound desc = could not find container \"18751b2634d75dd73d0a3ff900aa645f1509a1bc3f7abe21408fe63586adbf11\": container with ID starting with 18751b2634d75dd73d0a3ff900aa645f1509a1bc3f7abe21408fe63586adbf11 not found: ID does not exist"
May 28 01:19:11.706172 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.706106    2550 scope.go:117] "RemoveContainer" containerID="32bd4d56fe25d671f6db75440980e337e217ed6ea7e8b6b52b142e999d884262"
May 28 01:19:11.706351 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:19:11.706331    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32bd4d56fe25d671f6db75440980e337e217ed6ea7e8b6b52b142e999d884262\": container with ID starting with 32bd4d56fe25d671f6db75440980e337e217ed6ea7e8b6b52b142e999d884262 not found: ID does not exist" containerID="32bd4d56fe25d671f6db75440980e337e217ed6ea7e8b6b52b142e999d884262"
May 28 01:19:11.706515 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.706360    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32bd4d56fe25d671f6db75440980e337e217ed6ea7e8b6b52b142e999d884262"} err="failed to get container status \"32bd4d56fe25d671f6db75440980e337e217ed6ea7e8b6b52b142e999d884262\": rpc error: code = NotFound desc = could not find container \"32bd4d56fe25d671f6db75440980e337e217ed6ea7e8b6b52b142e999d884262\": container with ID starting with 32bd4d56fe25d671f6db75440980e337e217ed6ea7e8b6b52b142e999d884262 not found: ID does not exist"
May 28 01:19:11.706515 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.706384    2550 scope.go:117] "RemoveContainer" containerID="04a2d02e89224adc55eec5c9036677a398fda09ee496b6250baf79f0adcecca8"
May 28 01:19:11.706649 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:19:11.706634    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04a2d02e89224adc55eec5c9036677a398fda09ee496b6250baf79f0adcecca8\": container with ID starting with 04a2d02e89224adc55eec5c9036677a398fda09ee496b6250baf79f0adcecca8 not found: ID does not exist" containerID="04a2d02e89224adc55eec5c9036677a398fda09ee496b6250baf79f0adcecca8"
May 28 01:19:11.706698 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.706653    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04a2d02e89224adc55eec5c9036677a398fda09ee496b6250baf79f0adcecca8"} err="failed to get container status \"04a2d02e89224adc55eec5c9036677a398fda09ee496b6250baf79f0adcecca8\": rpc error: code = NotFound desc = could not find container \"04a2d02e89224adc55eec5c9036677a398fda09ee496b6250baf79f0adcecca8\": container with ID starting with 04a2d02e89224adc55eec5c9036677a398fda09ee496b6250baf79f0adcecca8 not found: ID does not exist"
May 28 01:19:11.706698 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.706666    2550 scope.go:117] "RemoveContainer" containerID="e531fd300441ea57ba57f7552c6a99395a544482d34d4b7fc46a2549c3400a93"
May 28 01:19:11.706895 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:19:11.706877    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e531fd300441ea57ba57f7552c6a99395a544482d34d4b7fc46a2549c3400a93\": container with ID starting with e531fd300441ea57ba57f7552c6a99395a544482d34d4b7fc46a2549c3400a93 not found: ID does not exist" containerID="e531fd300441ea57ba57f7552c6a99395a544482d34d4b7fc46a2549c3400a93"
May 28 01:19:11.706936 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.706901    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e531fd300441ea57ba57f7552c6a99395a544482d34d4b7fc46a2549c3400a93"} err="failed to get container status \"e531fd300441ea57ba57f7552c6a99395a544482d34d4b7fc46a2549c3400a93\": rpc error: code = NotFound desc = could not find container \"e531fd300441ea57ba57f7552c6a99395a544482d34d4b7fc46a2549c3400a93\": container with ID starting with e531fd300441ea57ba57f7552c6a99395a544482d34d4b7fc46a2549c3400a93 not found: ID does not exist"
May 28 01:19:11.706936 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.706916    2550 scope.go:117] "RemoveContainer" containerID="c29b27bfbe9a982f9737438bb947090ef19ce934655e5fd5d148ca851001459c"
May 28 01:19:11.707095 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.707075    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c29b27bfbe9a982f9737438bb947090ef19ce934655e5fd5d148ca851001459c"} err="failed to get container status \"c29b27bfbe9a982f9737438bb947090ef19ce934655e5fd5d148ca851001459c\": rpc error: code = NotFound desc = could not find container \"c29b27bfbe9a982f9737438bb947090ef19ce934655e5fd5d148ca851001459c\": container with ID starting with c29b27bfbe9a982f9737438bb947090ef19ce934655e5fd5d148ca851001459c not found: ID does not exist"
May 28 01:19:11.707140 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.707096    2550 scope.go:117] "RemoveContainer" containerID="98c84b838223011fb82bb5fdb84b4771ddfbd70757486f9ff7f40f9ed51c1983"
May 28 01:19:11.707282 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.707265    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98c84b838223011fb82bb5fdb84b4771ddfbd70757486f9ff7f40f9ed51c1983"} err="failed to get container status \"98c84b838223011fb82bb5fdb84b4771ddfbd70757486f9ff7f40f9ed51c1983\": rpc error: code = NotFound desc = could not find container \"98c84b838223011fb82bb5fdb84b4771ddfbd70757486f9ff7f40f9ed51c1983\": container with ID starting with 98c84b838223011fb82bb5fdb84b4771ddfbd70757486f9ff7f40f9ed51c1983 not found: ID does not exist"
May 28 01:19:11.707337 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.707282    2550 scope.go:117] "RemoveContainer" containerID="18751b2634d75dd73d0a3ff900aa645f1509a1bc3f7abe21408fe63586adbf11"
May 28 01:19:11.707469 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.707452    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18751b2634d75dd73d0a3ff900aa645f1509a1bc3f7abe21408fe63586adbf11"} err="failed to get container status \"18751b2634d75dd73d0a3ff900aa645f1509a1bc3f7abe21408fe63586adbf11\": rpc error: code = NotFound desc = could not find container \"18751b2634d75dd73d0a3ff900aa645f1509a1bc3f7abe21408fe63586adbf11\": container with ID starting with 18751b2634d75dd73d0a3ff900aa645f1509a1bc3f7abe21408fe63586adbf11 not found: ID does not exist"
May 28 01:19:11.707469 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.707468    2550 scope.go:117] "RemoveContainer" containerID="32bd4d56fe25d671f6db75440980e337e217ed6ea7e8b6b52b142e999d884262"
May 28 01:19:11.707695 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.707678    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32bd4d56fe25d671f6db75440980e337e217ed6ea7e8b6b52b142e999d884262"} err="failed to get container status \"32bd4d56fe25d671f6db75440980e337e217ed6ea7e8b6b52b142e999d884262\": rpc error: code = NotFound desc = could not find container \"32bd4d56fe25d671f6db75440980e337e217ed6ea7e8b6b52b142e999d884262\": container with ID starting with 32bd4d56fe25d671f6db75440980e337e217ed6ea7e8b6b52b142e999d884262 not found: ID does not exist"
May 28 01:19:11.707741 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.707695    2550 scope.go:117] "RemoveContainer" containerID="04a2d02e89224adc55eec5c9036677a398fda09ee496b6250baf79f0adcecca8"
May 28 01:19:11.707879 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.707864    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04a2d02e89224adc55eec5c9036677a398fda09ee496b6250baf79f0adcecca8"} err="failed to get container status \"04a2d02e89224adc55eec5c9036677a398fda09ee496b6250baf79f0adcecca8\": rpc error: code = NotFound desc = could not find container \"04a2d02e89224adc55eec5c9036677a398fda09ee496b6250baf79f0adcecca8\": container with ID starting with 04a2d02e89224adc55eec5c9036677a398fda09ee496b6250baf79f0adcecca8 not found: ID does not exist"
May 28 01:19:11.707924 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.707879    2550 scope.go:117] "RemoveContainer" containerID="e531fd300441ea57ba57f7552c6a99395a544482d34d4b7fc46a2549c3400a93"
May 28 01:19:11.708052 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.708036    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e531fd300441ea57ba57f7552c6a99395a544482d34d4b7fc46a2549c3400a93"} err="failed to get container status \"e531fd300441ea57ba57f7552c6a99395a544482d34d4b7fc46a2549c3400a93\": rpc error: code = NotFound desc = could not find container \"e531fd300441ea57ba57f7552c6a99395a544482d34d4b7fc46a2549c3400a93\": container with ID starting with e531fd300441ea57ba57f7552c6a99395a544482d34d4b7fc46a2549c3400a93 not found: ID does not exist"
May 28 01:19:11.708094 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.708052    2550 scope.go:117] "RemoveContainer" containerID="ac4af642d735283aecaadeeaa32becff08be6b765ae521e768cb75758fa45b90"
May 28 01:19:11.715204 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.715185    2550 scope.go:117] "RemoveContainer" containerID="58d9ed10be64f2137f90c2292cb4cd1b84ec993b4dc06748976dd2cd5b2ad78f"
May 28 01:19:11.721860 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.721838    2550 scope.go:117] "RemoveContainer" containerID="ffe5bed9ead47ec184a52e880f56670c334f94585c9c5e1881dcb26ea8a53425"
May 28 01:19:11.728208 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.728191    2550 scope.go:117] "RemoveContainer" containerID="2a4f0ab42f03f3d17f73319be84425e35a24858fc2e261f1c550690052c3412c"
May 28 01:19:11.734977 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.734954    2550 scope.go:117] "RemoveContainer" containerID="27cd38013d385c07f99dfa5f62c24eb10f3efc9f57728e81326f3743c98f9f24"
May 28 01:19:11.742093 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.742075    2550 scope.go:117] "RemoveContainer" containerID="ac4af642d735283aecaadeeaa32becff08be6b765ae521e768cb75758fa45b90"
May 28 01:19:11.742371 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:19:11.742355    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac4af642d735283aecaadeeaa32becff08be6b765ae521e768cb75758fa45b90\": container with ID starting with ac4af642d735283aecaadeeaa32becff08be6b765ae521e768cb75758fa45b90 not found: ID does not exist" containerID="ac4af642d735283aecaadeeaa32becff08be6b765ae521e768cb75758fa45b90"
May 28 01:19:11.742446 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.742380    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac4af642d735283aecaadeeaa32becff08be6b765ae521e768cb75758fa45b90"} err="failed to get container status \"ac4af642d735283aecaadeeaa32becff08be6b765ae521e768cb75758fa45b90\": rpc error: code = NotFound desc = could not find container \"ac4af642d735283aecaadeeaa32becff08be6b765ae521e768cb75758fa45b90\": container with ID starting with ac4af642d735283aecaadeeaa32becff08be6b765ae521e768cb75758fa45b90 not found: ID does not exist"
May 28 01:19:11.742446 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.742399    2550 scope.go:117] "RemoveContainer" containerID="58d9ed10be64f2137f90c2292cb4cd1b84ec993b4dc06748976dd2cd5b2ad78f"
May 28 01:19:11.742667 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:19:11.742650    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58d9ed10be64f2137f90c2292cb4cd1b84ec993b4dc06748976dd2cd5b2ad78f\": container with ID starting with 58d9ed10be64f2137f90c2292cb4cd1b84ec993b4dc06748976dd2cd5b2ad78f not found: ID does not exist" containerID="58d9ed10be64f2137f90c2292cb4cd1b84ec993b4dc06748976dd2cd5b2ad78f"
May 28 01:19:11.742710 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.742671    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58d9ed10be64f2137f90c2292cb4cd1b84ec993b4dc06748976dd2cd5b2ad78f"} err="failed to get container status \"58d9ed10be64f2137f90c2292cb4cd1b84ec993b4dc06748976dd2cd5b2ad78f\": rpc error: code = NotFound desc = could not find container \"58d9ed10be64f2137f90c2292cb4cd1b84ec993b4dc06748976dd2cd5b2ad78f\": container with ID starting with 58d9ed10be64f2137f90c2292cb4cd1b84ec993b4dc06748976dd2cd5b2ad78f not found: ID does not exist"
May 28 01:19:11.742710 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.742685    2550 scope.go:117] "RemoveContainer" containerID="ffe5bed9ead47ec184a52e880f56670c334f94585c9c5e1881dcb26ea8a53425"
May 28 01:19:11.742884 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:19:11.742869    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffe5bed9ead47ec184a52e880f56670c334f94585c9c5e1881dcb26ea8a53425\": container with ID starting with ffe5bed9ead47ec184a52e880f56670c334f94585c9c5e1881dcb26ea8a53425 not found: ID does not exist" containerID="ffe5bed9ead47ec184a52e880f56670c334f94585c9c5e1881dcb26ea8a53425"
May 28 01:19:11.742932 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.742886    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffe5bed9ead47ec184a52e880f56670c334f94585c9c5e1881dcb26ea8a53425"} err="failed to get container status \"ffe5bed9ead47ec184a52e880f56670c334f94585c9c5e1881dcb26ea8a53425\": rpc error: code = NotFound desc = could not find container \"ffe5bed9ead47ec184a52e880f56670c334f94585c9c5e1881dcb26ea8a53425\": container with ID starting with ffe5bed9ead47ec184a52e880f56670c334f94585c9c5e1881dcb26ea8a53425 not found: ID does not exist"
May 28 01:19:11.742932 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.742898    2550 scope.go:117] "RemoveContainer" containerID="2a4f0ab42f03f3d17f73319be84425e35a24858fc2e261f1c550690052c3412c"
May 28 01:19:11.743063 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:19:11.743047    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a4f0ab42f03f3d17f73319be84425e35a24858fc2e261f1c550690052c3412c\": container with ID starting with 2a4f0ab42f03f3d17f73319be84425e35a24858fc2e261f1c550690052c3412c not found: ID does not exist" containerID="2a4f0ab42f03f3d17f73319be84425e35a24858fc2e261f1c550690052c3412c"
May 28 01:19:11.743100 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.743066    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a4f0ab42f03f3d17f73319be84425e35a24858fc2e261f1c550690052c3412c"} err="failed to get container status \"2a4f0ab42f03f3d17f73319be84425e35a24858fc2e261f1c550690052c3412c\": rpc error: code = NotFound desc = could not find container \"2a4f0ab42f03f3d17f73319be84425e35a24858fc2e261f1c550690052c3412c\": container with ID starting with 2a4f0ab42f03f3d17f73319be84425e35a24858fc2e261f1c550690052c3412c not found: ID does not exist"
May 28 01:19:11.743100 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.743080    2550 scope.go:117] "RemoveContainer" containerID="27cd38013d385c07f99dfa5f62c24eb10f3efc9f57728e81326f3743c98f9f24"
May 28 01:19:11.743272 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:19:11.743256    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27cd38013d385c07f99dfa5f62c24eb10f3efc9f57728e81326f3743c98f9f24\": container with ID starting with 27cd38013d385c07f99dfa5f62c24eb10f3efc9f57728e81326f3743c98f9f24 not found: ID does not exist" containerID="27cd38013d385c07f99dfa5f62c24eb10f3efc9f57728e81326f3743c98f9f24"
May 28 01:19:11.743313 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.743276    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27cd38013d385c07f99dfa5f62c24eb10f3efc9f57728e81326f3743c98f9f24"} err="failed to get container status \"27cd38013d385c07f99dfa5f62c24eb10f3efc9f57728e81326f3743c98f9f24\": rpc error: code = NotFound desc = could not find container \"27cd38013d385c07f99dfa5f62c24eb10f3efc9f57728e81326f3743c98f9f24\": container with ID starting with 27cd38013d385c07f99dfa5f62c24eb10f3efc9f57728e81326f3743c98f9f24 not found: ID does not exist"
May 28 01:19:11.795258 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.795167    2550 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-offline-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:19:11.795258 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.795193    2550 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/00171e68-2770-4c85-8b03-3996f824109d-feast-data\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:19:11.795258 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.795202    2550 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-ui-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:19:11.795258 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.795211    2550 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-online-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:19:11.795258 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.795219    2550 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/00171e68-2770-4c85-8b03-3996f824109d-registry-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:19:11.795258 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.795228    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzsqn\" (UniqueName: \"kubernetes.io/projected/00171e68-2770-4c85-8b03-3996f824109d-kube-api-access-qzsqn\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:19:11.979593 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.979556    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"]
May 28 01:19:11.982481 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:11.982456    2550 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-txsd2"]
May 28 01:19:13.323701 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:13.323666    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00171e68-2770-4c85-8b03-3996f824109d" path="/var/lib/kubelet/pods/00171e68-2770-4c85-8b03-3996f824109d/volumes"
May 28 01:19:13.324285 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:13.324270    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d05ee48-ebbe-49ff-8632-1125083a8075" path="/var/lib/kubelet/pods/4d05ee48-ebbe-49ff-8632-1125083a8075/volumes"
May 28 01:19:53.147746 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.147710    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"]
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.147961    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerName="online"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.147971    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerName="online"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.147980    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerName="offline"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.147986    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerName="offline"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.147992    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerName="ui"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.147997    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerName="ui"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.148006    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="online"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.148012    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="online"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.148017    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerName="feast-apply"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.148023    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerName="feast-apply"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.148030    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="feast-init"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.148036    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="feast-init"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.148041    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="ui"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.148047    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="ui"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.148058    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="feast-apply"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.148063    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="feast-apply"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.148069    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerName="feast-init"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.148074    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerName="feast-init"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.148080    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="registry"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.148085    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="registry"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.148096    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="offline"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.148101    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="offline"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.148138    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerName="online"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.148146    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="registry"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.148152    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="offline"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.148159    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerName="offline"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.148165    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="online"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.148172    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="4d05ee48-ebbe-49ff-8632-1125083a8075" containerName="ui"
May 28 01:19:53.148291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.148179    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="00171e68-2770-4c85-8b03-3996f824109d" containerName="ui"
May 28 01:19:53.152750 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.152733    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:19:53.156227 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.156203    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 28 01:19:53.156579 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.156556    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 28 01:19:53.156579 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.156570    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 28 01:19:53.156779 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.156766    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 28 01:19:53.156833 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.156790    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 28 01:19:53.156833 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.156819    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-bhh4n\""
May 28 01:19:53.158183 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.158165    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 28 01:19:53.163120 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.163097    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"]
May 28 01:19:53.218244 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.218207    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-5rgg2\" (UID: \"77027281-bb37-42b8-9083-2daf2c20aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:19:53.218244 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.218245    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/77027281-bb37-42b8-9083-2daf2c20aa72-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-5rgg2\" (UID: \"77027281-bb37-42b8-9083-2daf2c20aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:19:53.218466 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.218323    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-5rgg2\" (UID: \"77027281-bb37-42b8-9083-2daf2c20aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:19:53.218466 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.218394    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-5rgg2\" (UID: \"77027281-bb37-42b8-9083-2daf2c20aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:19:53.218466 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.218439    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-5rgg2\" (UID: \"77027281-bb37-42b8-9083-2daf2c20aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:19:53.218562 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.218478    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfnlt\" (UniqueName: \"kubernetes.io/projected/77027281-bb37-42b8-9083-2daf2c20aa72-kube-api-access-xfnlt\") pod \"feast-simple-feast-setup-664fcfcc58-5rgg2\" (UID: \"77027281-bb37-42b8-9083-2daf2c20aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:19:53.318842 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.318805    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xfnlt\" (UniqueName: \"kubernetes.io/projected/77027281-bb37-42b8-9083-2daf2c20aa72-kube-api-access-xfnlt\") pod \"feast-simple-feast-setup-664fcfcc58-5rgg2\" (UID: \"77027281-bb37-42b8-9083-2daf2c20aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:19:53.319023 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.318857    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-5rgg2\" (UID: \"77027281-bb37-42b8-9083-2daf2c20aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:19:53.319023 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.318879    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/77027281-bb37-42b8-9083-2daf2c20aa72-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-5rgg2\" (UID: \"77027281-bb37-42b8-9083-2daf2c20aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:19:53.319023 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.318923    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-5rgg2\" (UID: \"77027281-bb37-42b8-9083-2daf2c20aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:19:53.319023 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.318960    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-5rgg2\" (UID: \"77027281-bb37-42b8-9083-2daf2c20aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:19:53.319224 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.319168    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-5rgg2\" (UID: \"77027281-bb37-42b8-9083-2daf2c20aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:19:53.319410 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:19:53.319379    2550 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found
May 28 01:19:53.319567 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.319406    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/77027281-bb37-42b8-9083-2daf2c20aa72-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-5rgg2\" (UID: \"77027281-bb37-42b8-9083-2daf2c20aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:19:53.319567 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:19:53.319512    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-offline-tls podName:77027281-bb37-42b8-9083-2daf2c20aa72 nodeName:}" failed. No retries permitted until 2026-05-28 01:19:53.819491327 +0000 UTC m=+1453.100448411 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-offline-tls") pod "feast-simple-feast-setup-664fcfcc58-5rgg2" (UID: "77027281-bb37-42b8-9083-2daf2c20aa72") : secret "feast-simple-feast-setup-offline-tls" not found
May 28 01:19:53.321683 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.321662    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-5rgg2\" (UID: \"77027281-bb37-42b8-9083-2daf2c20aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:19:53.321780 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.321664    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-5rgg2\" (UID: \"77027281-bb37-42b8-9083-2daf2c20aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:19:53.322062 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.322036    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-5rgg2\" (UID: \"77027281-bb37-42b8-9083-2daf2c20aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:19:53.327999 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.327973    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfnlt\" (UniqueName: \"kubernetes.io/projected/77027281-bb37-42b8-9083-2daf2c20aa72-kube-api-access-xfnlt\") pod \"feast-simple-feast-setup-664fcfcc58-5rgg2\" (UID: \"77027281-bb37-42b8-9083-2daf2c20aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:19:53.823362 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.823325    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-5rgg2\" (UID: \"77027281-bb37-42b8-9083-2daf2c20aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:19:53.825729 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:53.825710    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-5rgg2\" (UID: \"77027281-bb37-42b8-9083-2daf2c20aa72\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:19:54.073360 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:54.073307    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:19:54.205249 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:54.205222    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"]
May 28 01:19:54.207469 ip-10-0-131-73 kubenswrapper[2550]: W0528 01:19:54.207441    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77027281_bb37_42b8_9083_2daf2c20aa72.slice/crio-96521848db816cb39a70f191e1382ff207c57e1e3a6637dfc7967b2e4d3e1634 WatchSource:0}: Error finding container 96521848db816cb39a70f191e1382ff207c57e1e3a6637dfc7967b2e4d3e1634: Status 404 returned error can't find the container with id 96521848db816cb39a70f191e1382ff207c57e1e3a6637dfc7967b2e4d3e1634
May 28 01:19:54.209240 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:54.209217    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 01:19:54.781244 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:54.781212    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" event={"ID":"77027281-bb37-42b8-9083-2daf2c20aa72","Type":"ContainerStarted","Data":"6ee5def19b33e30e3eb41be8dfab7fc2fb7caa53c50e23c7e2ad4ca7f10d733b"}
May 28 01:19:54.781244 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:54.781247    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" event={"ID":"77027281-bb37-42b8-9083-2daf2c20aa72","Type":"ContainerStarted","Data":"96521848db816cb39a70f191e1382ff207c57e1e3a6637dfc7967b2e4d3e1634"}
May 28 01:19:59.796559 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:59.796523    2550 generic.go:358] "Generic (PLEG): container finished" podID="77027281-bb37-42b8-9083-2daf2c20aa72" containerID="6ee5def19b33e30e3eb41be8dfab7fc2fb7caa53c50e23c7e2ad4ca7f10d733b" exitCode=0
May 28 01:19:59.796928 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:19:59.796596    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" event={"ID":"77027281-bb37-42b8-9083-2daf2c20aa72","Type":"ContainerDied","Data":"6ee5def19b33e30e3eb41be8dfab7fc2fb7caa53c50e23c7e2ad4ca7f10d733b"}
May 28 01:20:00.801569 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:00.801533    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" event={"ID":"77027281-bb37-42b8-9083-2daf2c20aa72","Type":"ContainerStarted","Data":"e84684ab13f4a05fb8da650d572a7062936b2333905bb9f8dd8e464eb2907ccd"}
May 28 01:20:05.817545 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:05.817497    2550 generic.go:358] "Generic (PLEG): container finished" podID="77027281-bb37-42b8-9083-2daf2c20aa72" containerID="e84684ab13f4a05fb8da650d572a7062936b2333905bb9f8dd8e464eb2907ccd" exitCode=0
May 28 01:20:05.817912 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:05.817569    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" event={"ID":"77027281-bb37-42b8-9083-2daf2c20aa72","Type":"ContainerDied","Data":"e84684ab13f4a05fb8da650d572a7062936b2333905bb9f8dd8e464eb2907ccd"}
May 28 01:20:06.825600 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:06.825553    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" event={"ID":"77027281-bb37-42b8-9083-2daf2c20aa72","Type":"ContainerStarted","Data":"66544252c6bb70bc82489237a6fab1eb50a535ac03878a19f6564139ee69bf0e"}
May 28 01:20:06.825600 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:06.825604    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" event={"ID":"77027281-bb37-42b8-9083-2daf2c20aa72","Type":"ContainerStarted","Data":"ca05827824217c619c5ef8d962476ccb0aba661f5b90d1dca4f5b4d0cd39c4d9"}
May 28 01:20:06.826146 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:06.825619    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" event={"ID":"77027281-bb37-42b8-9083-2daf2c20aa72","Type":"ContainerStarted","Data":"0fd640467229060c5ee19a04d415da76204c5d20cbc2607ba924449d99bce1e8"}
May 28 01:20:06.826146 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:06.825634    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" event={"ID":"77027281-bb37-42b8-9083-2daf2c20aa72","Type":"ContainerStarted","Data":"4f6248375685fe4ef6da575ca46e342ae60272dfcd775c534fa940dd03f41d06"}
May 28 01:20:06.851642 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:06.851571    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" podStartSLOduration=13.851550196 podStartE2EDuration="13.851550196s" podCreationTimestamp="2026-05-28 01:19:53 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 01:20:06.849936998 +0000 UTC m=+1466.130894093" watchObservedRunningTime="2026-05-28 01:20:06.851550196 +0000 UTC m=+1466.132507290"
May 28 01:20:09.074499 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:09.074463    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:20:09.075064 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:09.075043    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:20:09.075203 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:09.075181    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:20:09.075203 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:09.075206    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:20:09.076605 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:09.076466    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.32:8443: connect: connection refused"
May 28 01:20:09.076605 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:09.076480    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.32:6571: connect: connection refused"
May 28 01:20:09.076605 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:09.076538    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.32:8816: connect: connection refused"
May 28 01:20:09.076605 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:09.076594    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused"
May 28 01:20:12.075491 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:12.075446    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:20:12.075987 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:12.075594    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:20:12.075987 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:12.075873    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:20:12.076146 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:12.076129    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:20:12.076190 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:12.076156    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:20:12.076190 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:12.076168    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:20:12.076333 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:12.076315    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:20:12.076503 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:12.076480    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:20:12.076603 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:12.076534    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:20:12.080383 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:12.080360    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:20:12.844768 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:12.844740    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:20:12.847806 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:12.847780    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:20:41.306474 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:41.306443    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9z98m_6671c1c1-b674-4a1d-9011-1ca22724b234/ovn-acl-logging/0.log"
May 28 01:20:41.313121 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:20:41.313099    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9z98m_6671c1c1-b674-4a1d-9011-1ca22724b234/ovn-acl-logging/0.log"
May 28 01:21:13.933181 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:13.933090    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"]
May 28 01:21:13.933726 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:13.933600    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="registry" containerID="cri-o://4f6248375685fe4ef6da575ca46e342ae60272dfcd775c534fa940dd03f41d06" gracePeriod=30
May 28 01:21:13.933726 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:13.933680    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="offline" containerID="cri-o://ca05827824217c619c5ef8d962476ccb0aba661f5b90d1dca4f5b4d0cd39c4d9" gracePeriod=30
May 28 01:21:13.933851 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:13.933722    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="online" containerID="cri-o://0fd640467229060c5ee19a04d415da76204c5d20cbc2607ba924449d99bce1e8" gracePeriod=30
May 28 01:21:13.933851 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:13.933718    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="ui" containerID="cri-o://66544252c6bb70bc82489237a6fab1eb50a535ac03878a19f6564139ee69bf0e" gracePeriod=30
May 28 01:21:14.250808 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:14.250714    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"]
May 28 01:21:14.254463 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:14.254444    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:14.256744 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:14.256724    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-pwxmw\""
May 28 01:21:14.266719 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:14.266691    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"]
May 28 01:21:14.281923 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:14.281890    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v24bk\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:14.282119 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:14.281941    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v24bk\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:14.282119 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:14.281982    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v24bk\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:14.282119 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:14.282003    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvmd7\" (UniqueName: \"kubernetes.io/projected/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-kube-api-access-wvmd7\") pod \"feast-simple-feast-setup-664fcfcc58-v24bk\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:14.282119 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:14.282020    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v24bk\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:14.282119 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:14.282110    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-v24bk\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:14.382786 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:14.382757    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v24bk\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:14.382907 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:14.382790    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wvmd7\" (UniqueName: \"kubernetes.io/projected/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-kube-api-access-wvmd7\") pod \"feast-simple-feast-setup-664fcfcc58-v24bk\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:14.382907 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:14.382812    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v24bk\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:14.382907 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:14.382869    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-v24bk\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:14.383050 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:21:14.382918    2550 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 28 01:21:14.383050 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:21:14.382977    2550 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found
May 28 01:21:14.383050 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:21:14.383002    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-ui-tls podName:b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9 nodeName:}" failed. No retries permitted until 2026-05-28 01:21:14.882977249 +0000 UTC m=+1534.163934336 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-ui-tls") pod "feast-simple-feast-setup-664fcfcc58-v24bk" (UID: "b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9") : secret "feast-simple-feast-setup-ui-tls" not found
May 28 01:21:14.383050 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:21:14.383044    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-registry-tls podName:b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9 nodeName:}" failed. No retries permitted until 2026-05-28 01:21:14.883027907 +0000 UTC m=+1534.163984985 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-registry-tls") pod "feast-simple-feast-setup-664fcfcc58-v24bk" (UID: "b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9") : secret "feast-simple-feast-setup-registry-tls" not found
May 28 01:21:14.383231 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:14.382923    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v24bk\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:14.383282 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:14.383241    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v24bk\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:14.383370 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:21:14.383353    2550 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found
May 28 01:21:14.383438 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:14.383350    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-v24bk\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:14.383438 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:21:14.383432    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-offline-tls podName:b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9 nodeName:}" failed. No retries permitted until 2026-05-28 01:21:14.883401924 +0000 UTC m=+1534.164358994 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-offline-tls") pod "feast-simple-feast-setup-664fcfcc58-v24bk" (UID: "b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9") : secret "feast-simple-feast-setup-offline-tls" not found
May 28 01:21:14.385291 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:14.385267    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v24bk\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:14.390878 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:14.390849    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvmd7\" (UniqueName: \"kubernetes.io/projected/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-kube-api-access-wvmd7\") pod \"feast-simple-feast-setup-664fcfcc58-v24bk\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:14.887310 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:14.887200    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v24bk\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:14.887310 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:14.887269    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v24bk\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:14.887310 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:14.887298    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v24bk\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:14.889624 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:14.889600    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v24bk\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:14.889761 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:14.889738    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v24bk\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:14.889804 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:14.889742    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-v24bk\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:15.012207 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:15.012175    2550 generic.go:358] "Generic (PLEG): container finished" podID="77027281-bb37-42b8-9083-2daf2c20aa72" containerID="66544252c6bb70bc82489237a6fab1eb50a535ac03878a19f6564139ee69bf0e" exitCode=0
May 28 01:21:15.012207 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:15.012199    2550 generic.go:358] "Generic (PLEG): container finished" podID="77027281-bb37-42b8-9083-2daf2c20aa72" containerID="0fd640467229060c5ee19a04d415da76204c5d20cbc2607ba924449d99bce1e8" exitCode=0
May 28 01:21:15.012614 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:15.012239    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" event={"ID":"77027281-bb37-42b8-9083-2daf2c20aa72","Type":"ContainerDied","Data":"66544252c6bb70bc82489237a6fab1eb50a535ac03878a19f6564139ee69bf0e"}
May 28 01:21:15.012614 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:15.012273    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" event={"ID":"77027281-bb37-42b8-9083-2daf2c20aa72","Type":"ContainerDied","Data":"0fd640467229060c5ee19a04d415da76204c5d20cbc2607ba924449d99bce1e8"}
May 28 01:21:15.164665 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:15.164640    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:15.292921 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:15.292891    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"]
May 28 01:21:15.296383 ip-10-0-131-73 kubenswrapper[2550]: W0528 01:21:15.296353    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb7c2b0b3_9ea2_4129_bdce_8e7d69d215b9.slice/crio-1e248eaabbbf1a1018016c3d94ffcec0c3633d1ba66cf5d43dfdf44fcc6ea8e9 WatchSource:0}: Error finding container 1e248eaabbbf1a1018016c3d94ffcec0c3633d1ba66cf5d43dfdf44fcc6ea8e9: Status 404 returned error can't find the container with id 1e248eaabbbf1a1018016c3d94ffcec0c3633d1ba66cf5d43dfdf44fcc6ea8e9
May 28 01:21:16.016056 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:16.016018    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" event={"ID":"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9","Type":"ContainerStarted","Data":"db9b2dcecd98cec9ec1ecdf2a1e668389ecb085342dec6d65048ce906e84b0b7"}
May 28 01:21:16.016056 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:16.016058    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" event={"ID":"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9","Type":"ContainerStarted","Data":"1e248eaabbbf1a1018016c3d94ffcec0c3633d1ba66cf5d43dfdf44fcc6ea8e9"}
May 28 01:21:20.028933 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:20.028898    2550 generic.go:358] "Generic (PLEG): container finished" podID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerID="db9b2dcecd98cec9ec1ecdf2a1e668389ecb085342dec6d65048ce906e84b0b7" exitCode=0
May 28 01:21:20.029333 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:20.028974    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" event={"ID":"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9","Type":"ContainerDied","Data":"db9b2dcecd98cec9ec1ecdf2a1e668389ecb085342dec6d65048ce906e84b0b7"}
May 28 01:21:21.033530 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:21.033491    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" event={"ID":"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9","Type":"ContainerStarted","Data":"12f11adef9439b0b5d3f0a1e751ca58dfe1f4cff6b96131018d4e54303f357f6"}
May 28 01:21:22.076370 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:22.076323    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.32:8443: connect: connection refused"
May 28 01:21:22.845400 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:22.845350    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused"
May 28 01:21:26.049646 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:26.049613    2550 generic.go:358] "Generic (PLEG): container finished" podID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerID="12f11adef9439b0b5d3f0a1e751ca58dfe1f4cff6b96131018d4e54303f357f6" exitCode=0
May 28 01:21:26.050024 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:26.049666    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" event={"ID":"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9","Type":"ContainerDied","Data":"12f11adef9439b0b5d3f0a1e751ca58dfe1f4cff6b96131018d4e54303f357f6"}
May 28 01:21:27.056894 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:27.056848    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" event={"ID":"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9","Type":"ContainerStarted","Data":"5f00be2a4bf5cd98cbb40592515222e4375032a3bda98ff2b6a3727fbe4d2427"}
May 28 01:21:27.056894 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:27.056901    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" event={"ID":"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9","Type":"ContainerStarted","Data":"fb7b0f48be3aa87281e343006ec551b1b244c7c1365e437839cb4ea5540732c1"}
May 28 01:21:27.057443 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:27.056915    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" event={"ID":"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9","Type":"ContainerStarted","Data":"2e1f569fa42ff971250cb6e22bcc9db97c12890fdd024ad187d391381acca140"}
May 28 01:21:27.057443 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:27.056928    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" event={"ID":"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9","Type":"ContainerStarted","Data":"2a6136880471f966868c8525b08161f83e5e1867b70dd04789da60a1530a0398"}
May 28 01:21:27.079333 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:27.079269    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" podStartSLOduration=13.079249178 podStartE2EDuration="13.079249178s" podCreationTimestamp="2026-05-28 01:21:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 01:21:27.077761707 +0000 UTC m=+1546.358718827" watchObservedRunningTime="2026-05-28 01:21:27.079249178 +0000 UTC m=+1546.360206273"
May 28 01:21:27.165265 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:27.165212    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:27.165265 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:27.165272    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:27.165565 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:27.165291    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:27.165565 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:27.165305    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:27.167492 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:27.167448    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.33:6571: connect: connection refused"
May 28 01:21:27.167634 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:27.167488    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.33:8816: connect: connection refused"
May 28 01:21:27.167634 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:27.167437    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.33:8443: connect: connection refused"
May 28 01:21:27.167768 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:27.167678    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused"
May 28 01:21:30.165032 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:30.164989    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.33:6571: connect: connection refused"
May 28 01:21:30.165655 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:30.164997    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused"
May 28 01:21:30.165792 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:30.164994    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.33:8816: connect: connection refused"
May 28 01:21:30.165792 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:30.164996    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.33:8443: connect: connection refused"
May 28 01:21:32.076677 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:32.076632    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.32:8443: connect: connection refused"
May 28 01:21:32.845183 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:32.845136    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused"
May 28 01:21:33.166573 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:33.166545    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:33.166944 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:33.166639    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:33.166944 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:33.166921    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:33.167366 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:33.167341    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:33.167366 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:33.167370    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:33.167664 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:33.167384    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:33.167719 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:33.167660    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:33.167719 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:33.167711    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:33.167872 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:33.167856    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:33.171518 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:33.171500    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:34.082209 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:34.082175    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:34.085229 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:34.085206    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:21:42.076325 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:42.076273    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.32:8443: connect: connection refused"
May 28 01:21:42.078763 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:42.076451    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:21:42.845809 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:42.845761    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused"
May 28 01:21:42.846019 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:42.845884    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:21:44.114803 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:44.114764    2550 generic.go:358] "Generic (PLEG): container finished" podID="77027281-bb37-42b8-9083-2daf2c20aa72" containerID="ca05827824217c619c5ef8d962476ccb0aba661f5b90d1dca4f5b4d0cd39c4d9" exitCode=137
May 28 01:21:44.114803 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:44.114796    2550 generic.go:358] "Generic (PLEG): container finished" podID="77027281-bb37-42b8-9083-2daf2c20aa72" containerID="4f6248375685fe4ef6da575ca46e342ae60272dfcd775c534fa940dd03f41d06" exitCode=137
May 28 01:21:44.115202 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:44.114836    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" event={"ID":"77027281-bb37-42b8-9083-2daf2c20aa72","Type":"ContainerDied","Data":"ca05827824217c619c5ef8d962476ccb0aba661f5b90d1dca4f5b4d0cd39c4d9"}
May 28 01:21:44.115202 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:44.114869    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" event={"ID":"77027281-bb37-42b8-9083-2daf2c20aa72","Type":"ContainerDied","Data":"4f6248375685fe4ef6da575ca46e342ae60272dfcd775c534fa940dd03f41d06"}
May 28 01:21:44.582249 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:44.582223    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:21:44.649843 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:44.649802    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-registry-tls\") pod \"77027281-bb37-42b8-9083-2daf2c20aa72\" (UID: \"77027281-bb37-42b8-9083-2daf2c20aa72\") "
May 28 01:21:44.650026 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:44.649856    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/77027281-bb37-42b8-9083-2daf2c20aa72-feast-data\") pod \"77027281-bb37-42b8-9083-2daf2c20aa72\" (UID: \"77027281-bb37-42b8-9083-2daf2c20aa72\") "
May 28 01:21:44.650026 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:44.649902    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfnlt\" (UniqueName: \"kubernetes.io/projected/77027281-bb37-42b8-9083-2daf2c20aa72-kube-api-access-xfnlt\") pod \"77027281-bb37-42b8-9083-2daf2c20aa72\" (UID: \"77027281-bb37-42b8-9083-2daf2c20aa72\") "
May 28 01:21:44.650026 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:44.649951    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-ui-tls\") pod \"77027281-bb37-42b8-9083-2daf2c20aa72\" (UID: \"77027281-bb37-42b8-9083-2daf2c20aa72\") "
May 28 01:21:44.650026 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:44.649975    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-online-tls\") pod \"77027281-bb37-42b8-9083-2daf2c20aa72\" (UID: \"77027281-bb37-42b8-9083-2daf2c20aa72\") "
May 28 01:21:44.650026 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:44.650010    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-offline-tls\") pod \"77027281-bb37-42b8-9083-2daf2c20aa72\" (UID: \"77027281-bb37-42b8-9083-2daf2c20aa72\") "
May 28 01:21:44.650527 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:44.650497    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77027281-bb37-42b8-9083-2daf2c20aa72-feast-data" (OuterVolumeSpecName: "feast-data") pod "77027281-bb37-42b8-9083-2daf2c20aa72" (UID: "77027281-bb37-42b8-9083-2daf2c20aa72"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 01:21:44.652245 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:44.652186    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "77027281-bb37-42b8-9083-2daf2c20aa72" (UID: "77027281-bb37-42b8-9083-2daf2c20aa72"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:21:44.652348 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:44.652240    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-online-tls" (OuterVolumeSpecName: "online-tls") pod "77027281-bb37-42b8-9083-2daf2c20aa72" (UID: "77027281-bb37-42b8-9083-2daf2c20aa72"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:21:44.652348 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:44.652259    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "77027281-bb37-42b8-9083-2daf2c20aa72" (UID: "77027281-bb37-42b8-9083-2daf2c20aa72"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:21:44.652468 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:44.652449    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77027281-bb37-42b8-9083-2daf2c20aa72-kube-api-access-xfnlt" (OuterVolumeSpecName: "kube-api-access-xfnlt") pod "77027281-bb37-42b8-9083-2daf2c20aa72" (UID: "77027281-bb37-42b8-9083-2daf2c20aa72"). InnerVolumeSpecName "kube-api-access-xfnlt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 01:21:44.652524 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:44.652504    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "77027281-bb37-42b8-9083-2daf2c20aa72" (UID: "77027281-bb37-42b8-9083-2daf2c20aa72"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:21:44.751310 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:44.751192    2550 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-ui-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:21:44.751310 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:44.751241    2550 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-online-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:21:44.751310 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:44.751257    2550 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-offline-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:21:44.751310 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:44.751268    2550 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/77027281-bb37-42b8-9083-2daf2c20aa72-registry-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:21:44.751310 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:44.751279    2550 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/77027281-bb37-42b8-9083-2daf2c20aa72-feast-data\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:21:44.751310 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:44.751291    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfnlt\" (UniqueName: \"kubernetes.io/projected/77027281-bb37-42b8-9083-2daf2c20aa72-kube-api-access-xfnlt\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:21:45.119919 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:45.119833    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2" event={"ID":"77027281-bb37-42b8-9083-2daf2c20aa72","Type":"ContainerDied","Data":"96521848db816cb39a70f191e1382ff207c57e1e3a6637dfc7967b2e4d3e1634"}
May 28 01:21:45.119919 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:45.119858    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"
May 28 01:21:45.119919 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:45.119887    2550 scope.go:117] "RemoveContainer" containerID="66544252c6bb70bc82489237a6fab1eb50a535ac03878a19f6564139ee69bf0e"
May 28 01:21:45.129299 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:45.129278    2550 scope.go:117] "RemoveContainer" containerID="ca05827824217c619c5ef8d962476ccb0aba661f5b90d1dca4f5b4d0cd39c4d9"
May 28 01:21:45.136624 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:45.136604    2550 scope.go:117] "RemoveContainer" containerID="0fd640467229060c5ee19a04d415da76204c5d20cbc2607ba924449d99bce1e8"
May 28 01:21:45.142306 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:45.142280    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"]
May 28 01:21:45.145261 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:45.145241    2550 scope.go:117] "RemoveContainer" containerID="4f6248375685fe4ef6da575ca46e342ae60272dfcd775c534fa940dd03f41d06"
May 28 01:21:45.146273 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:45.146249    2550 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-5rgg2"]
May 28 01:21:45.152499 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:45.152475    2550 scope.go:117] "RemoveContainer" containerID="e84684ab13f4a05fb8da650d572a7062936b2333905bb9f8dd8e464eb2907ccd"
May 28 01:21:45.159373 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:45.159354    2550 scope.go:117] "RemoveContainer" containerID="6ee5def19b33e30e3eb41be8dfab7fc2fb7caa53c50e23c7e2ad4ca7f10d733b"
May 28 01:21:45.323973 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:21:45.323940    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" path="/var/lib/kubelet/pods/77027281-bb37-42b8-9083-2daf2c20aa72/volumes"
May 28 01:22:35.738357 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.738274    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"]
May 28 01:22:35.738850 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.738551    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="ui"
May 28 01:22:35.738850 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.738562    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="ui"
May 28 01:22:35.738850 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.738573    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="feast-apply"
May 28 01:22:35.738850 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.738580    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="feast-apply"
May 28 01:22:35.738850 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.738586    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="online"
May 28 01:22:35.738850 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.738592    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="online"
May 28 01:22:35.738850 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.738602    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="offline"
May 28 01:22:35.738850 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.738607    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="offline"
May 28 01:22:35.738850 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.738615    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="feast-init"
May 28 01:22:35.738850 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.738619    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="feast-init"
May 28 01:22:35.738850 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.738624    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="registry"
May 28 01:22:35.738850 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.738629    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="registry"
May 28 01:22:35.738850 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.738670    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="ui"
May 28 01:22:35.738850 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.738679    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="online"
May 28 01:22:35.738850 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.738685    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="offline"
May 28 01:22:35.738850 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.738691    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="77027281-bb37-42b8-9083-2daf2c20aa72" containerName="registry"
May 28 01:22:35.741749 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.741730    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:35.743747 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.743720    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 28 01:22:35.744829 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.744802    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 28 01:22:35.744952 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.744812    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 28 01:22:35.744952 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.744855    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 28 01:22:35.744952 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.744939    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-xdj99\""
May 28 01:22:35.745187 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.745173    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 28 01:22:35.745246 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.745225    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 28 01:22:35.755048 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.755018    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"]
May 28 01:22:35.862266 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.862229    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/adaf9acc-e950-41ab-9807-5a0b945b407d-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-c2f85\" (UID: \"adaf9acc-e950-41ab-9807-5a0b945b407d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:35.862266 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.862268    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/adaf9acc-e950-41ab-9807-5a0b945b407d-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-c2f85\" (UID: \"adaf9acc-e950-41ab-9807-5a0b945b407d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:35.862527 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.862328    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/adaf9acc-e950-41ab-9807-5a0b945b407d-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-c2f85\" (UID: \"adaf9acc-e950-41ab-9807-5a0b945b407d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:35.862527 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.862370    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/adaf9acc-e950-41ab-9807-5a0b945b407d-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-c2f85\" (UID: \"adaf9acc-e950-41ab-9807-5a0b945b407d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:35.862527 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.862451    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/adaf9acc-e950-41ab-9807-5a0b945b407d-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-c2f85\" (UID: \"adaf9acc-e950-41ab-9807-5a0b945b407d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:35.862527 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.862494    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wcc6\" (UniqueName: \"kubernetes.io/projected/adaf9acc-e950-41ab-9807-5a0b945b407d-kube-api-access-6wcc6\") pod \"feast-simple-feast-remote-setup-5cff94746b-c2f85\" (UID: \"adaf9acc-e950-41ab-9807-5a0b945b407d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:35.963563 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.963529    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/adaf9acc-e950-41ab-9807-5a0b945b407d-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-c2f85\" (UID: \"adaf9acc-e950-41ab-9807-5a0b945b407d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:35.963653 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.963585    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/adaf9acc-e950-41ab-9807-5a0b945b407d-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-c2f85\" (UID: \"adaf9acc-e950-41ab-9807-5a0b945b407d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:35.963653 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.963607    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6wcc6\" (UniqueName: \"kubernetes.io/projected/adaf9acc-e950-41ab-9807-5a0b945b407d-kube-api-access-6wcc6\") pod \"feast-simple-feast-remote-setup-5cff94746b-c2f85\" (UID: \"adaf9acc-e950-41ab-9807-5a0b945b407d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:35.963653 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.963632    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/adaf9acc-e950-41ab-9807-5a0b945b407d-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-c2f85\" (UID: \"adaf9acc-e950-41ab-9807-5a0b945b407d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:35.963804 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.963655    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/adaf9acc-e950-41ab-9807-5a0b945b407d-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-c2f85\" (UID: \"adaf9acc-e950-41ab-9807-5a0b945b407d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:35.963804 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.963682    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/adaf9acc-e950-41ab-9807-5a0b945b407d-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-c2f85\" (UID: \"adaf9acc-e950-41ab-9807-5a0b945b407d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:35.964155 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.964123    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/adaf9acc-e950-41ab-9807-5a0b945b407d-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-c2f85\" (UID: \"adaf9acc-e950-41ab-9807-5a0b945b407d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:35.964443 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.964401    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/adaf9acc-e950-41ab-9807-5a0b945b407d-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-c2f85\" (UID: \"adaf9acc-e950-41ab-9807-5a0b945b407d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:35.966126 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.966099    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/adaf9acc-e950-41ab-9807-5a0b945b407d-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-c2f85\" (UID: \"adaf9acc-e950-41ab-9807-5a0b945b407d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:35.966126 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.966117    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/adaf9acc-e950-41ab-9807-5a0b945b407d-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-c2f85\" (UID: \"adaf9acc-e950-41ab-9807-5a0b945b407d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:35.966253 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.966154    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/adaf9acc-e950-41ab-9807-5a0b945b407d-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-c2f85\" (UID: \"adaf9acc-e950-41ab-9807-5a0b945b407d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:35.970825 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:35.970807    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wcc6\" (UniqueName: \"kubernetes.io/projected/adaf9acc-e950-41ab-9807-5a0b945b407d-kube-api-access-6wcc6\") pod \"feast-simple-feast-remote-setup-5cff94746b-c2f85\" (UID: \"adaf9acc-e950-41ab-9807-5a0b945b407d\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:36.051318 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:36.051219    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:36.179908 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:36.179801    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"]
May 28 01:22:36.182562 ip-10-0-131-73 kubenswrapper[2550]: W0528 01:22:36.182532    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podadaf9acc_e950_41ab_9807_5a0b945b407d.slice/crio-a8ac3c03f6fa65a2d4a0d9a7e8b84777b8a30fcfebbb7cd19e3cf08999ad40d6 WatchSource:0}: Error finding container a8ac3c03f6fa65a2d4a0d9a7e8b84777b8a30fcfebbb7cd19e3cf08999ad40d6: Status 404 returned error can't find the container with id a8ac3c03f6fa65a2d4a0d9a7e8b84777b8a30fcfebbb7cd19e3cf08999ad40d6
May 28 01:22:36.263845 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:36.263810    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85" event={"ID":"adaf9acc-e950-41ab-9807-5a0b945b407d","Type":"ContainerStarted","Data":"5765c903ceebe83e9f95d7ba3471aee416e13c9a9f2f70cd443a43eacda17911"}
May 28 01:22:36.264013 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:36.263854    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85" event={"ID":"adaf9acc-e950-41ab-9807-5a0b945b407d","Type":"ContainerStarted","Data":"a8ac3c03f6fa65a2d4a0d9a7e8b84777b8a30fcfebbb7cd19e3cf08999ad40d6"}
May 28 01:22:41.280423 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:41.280389    2550 generic.go:358] "Generic (PLEG): container finished" podID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerID="5765c903ceebe83e9f95d7ba3471aee416e13c9a9f2f70cd443a43eacda17911" exitCode=0
May 28 01:22:41.280796 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:41.280464    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85" event={"ID":"adaf9acc-e950-41ab-9807-5a0b945b407d","Type":"ContainerDied","Data":"5765c903ceebe83e9f95d7ba3471aee416e13c9a9f2f70cd443a43eacda17911"}
May 28 01:22:42.285667 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:42.285632    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85" event={"ID":"adaf9acc-e950-41ab-9807-5a0b945b407d","Type":"ContainerStarted","Data":"703f2bfee918a2cf3c60bb6a2847155df56b164c1dced8d32dd2dd8656670ee5"}
May 28 01:22:47.301204 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:47.301115    2550 generic.go:358] "Generic (PLEG): container finished" podID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerID="703f2bfee918a2cf3c60bb6a2847155df56b164c1dced8d32dd2dd8656670ee5" exitCode=0
May 28 01:22:47.301204 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:47.301185    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85" event={"ID":"adaf9acc-e950-41ab-9807-5a0b945b407d","Type":"ContainerDied","Data":"703f2bfee918a2cf3c60bb6a2847155df56b164c1dced8d32dd2dd8656670ee5"}
May 28 01:22:48.307577 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:48.307537    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85" event={"ID":"adaf9acc-e950-41ab-9807-5a0b945b407d","Type":"ContainerStarted","Data":"35ca1687df3417aa0cbf8241202757629269c5eb11ffd687846da029ed2cd5ec"}
May 28 01:22:48.307577 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:48.307580    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85" event={"ID":"adaf9acc-e950-41ab-9807-5a0b945b407d","Type":"ContainerStarted","Data":"c409abca438527e6018583dc2d145548e48259b3fcb5c02d5f14f3467361b3d2"}
May 28 01:22:48.308158 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:48.307590    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85" event={"ID":"adaf9acc-e950-41ab-9807-5a0b945b407d","Type":"ContainerStarted","Data":"ed53ecd492baa0000e58140f4a261071abd57c24c9e86a6716a14c3ddc43012e"}
May 28 01:22:48.336119 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:48.336057    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85" podStartSLOduration=13.336039777 podStartE2EDuration="13.336039777s" podCreationTimestamp="2026-05-28 01:22:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 01:22:48.334185735 +0000 UTC m=+1627.615142829" watchObservedRunningTime="2026-05-28 01:22:48.336039777 +0000 UTC m=+1627.616996871"
May 28 01:22:51.052478 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:51.052394    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:51.052478 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:51.052465    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:51.052478 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:51.052480    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:51.054058 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:51.054026    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85" podUID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.34:8443: connect: connection refused"
May 28 01:22:51.054208 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:51.054068    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85" podUID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.34:6567/health\": dial tcp 10.133.0.34:6567: connect: connection refused"
May 28 01:22:51.054208 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:51.054028    2550 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85" podUID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.34:8816: connect: connection refused"
May 28 01:22:54.052714 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:54.052681    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:54.053181 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:54.052793    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:54.053264 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:54.053241    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:54.053318 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:54.053273    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:54.053479 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:54.053462    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:54.053665 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:54.053648    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:54.056883 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:54.056865    2550 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:54.326897 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:54.326814    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:22:54.329810 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:22:54.329792    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:23:56.619912 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:23:56.619831    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"]
May 28 01:23:56.620399 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:23:56.620172    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85" podUID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerName="online" containerID="cri-o://ed53ecd492baa0000e58140f4a261071abd57c24c9e86a6716a14c3ddc43012e" gracePeriod=30
May 28 01:23:56.620399 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:23:56.620231    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85" podUID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerName="offline" containerID="cri-o://c409abca438527e6018583dc2d145548e48259b3fcb5c02d5f14f3467361b3d2" gracePeriod=30
May 28 01:23:56.620399 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:23:56.620235    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85" podUID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerName="ui" containerID="cri-o://35ca1687df3417aa0cbf8241202757629269c5eb11ffd687846da029ed2cd5ec" gracePeriod=30
May 28 01:23:56.746099 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:23:56.746045    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"]
May 28 01:23:56.746466 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:23:56.746435    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="registry" containerID="cri-o://2a6136880471f966868c8525b08161f83e5e1867b70dd04789da60a1530a0398" gracePeriod=30
May 28 01:23:56.746561 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:23:56.746491    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="online" containerID="cri-o://2e1f569fa42ff971250cb6e22bcc9db97c12890fdd024ad187d391381acca140" gracePeriod=30
May 28 01:23:56.746561 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:23:56.746501    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="ui" containerID="cri-o://5f00be2a4bf5cd98cbb40592515222e4375032a3bda98ff2b6a3727fbe4d2427" gracePeriod=30
May 28 01:23:56.746693 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:23:56.746676    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="offline" containerID="cri-o://fb7b0f48be3aa87281e343006ec551b1b244c7c1365e437839cb4ea5540732c1" gracePeriod=30
May 28 01:23:57.498889 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:23:57.498856    2550 generic.go:358] "Generic (PLEG): container finished" podID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerID="5f00be2a4bf5cd98cbb40592515222e4375032a3bda98ff2b6a3727fbe4d2427" exitCode=0
May 28 01:23:57.498889 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:23:57.498884    2550 generic.go:358] "Generic (PLEG): container finished" podID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerID="2e1f569fa42ff971250cb6e22bcc9db97c12890fdd024ad187d391381acca140" exitCode=0
May 28 01:23:57.499091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:23:57.498927    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" event={"ID":"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9","Type":"ContainerDied","Data":"5f00be2a4bf5cd98cbb40592515222e4375032a3bda98ff2b6a3727fbe4d2427"}
May 28 01:23:57.499091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:23:57.498968    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" event={"ID":"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9","Type":"ContainerDied","Data":"2e1f569fa42ff971250cb6e22bcc9db97c12890fdd024ad187d391381acca140"}
May 28 01:23:57.501130 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:23:57.501110    2550 generic.go:358] "Generic (PLEG): container finished" podID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerID="35ca1687df3417aa0cbf8241202757629269c5eb11ffd687846da029ed2cd5ec" exitCode=0
May 28 01:23:57.501130 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:23:57.501128    2550 generic.go:358] "Generic (PLEG): container finished" podID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerID="ed53ecd492baa0000e58140f4a261071abd57c24c9e86a6716a14c3ddc43012e" exitCode=0
May 28 01:23:57.501266 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:23:57.501179    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85" event={"ID":"adaf9acc-e950-41ab-9807-5a0b945b407d","Type":"ContainerDied","Data":"35ca1687df3417aa0cbf8241202757629269c5eb11ffd687846da029ed2cd5ec"}
May 28 01:23:57.501266 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:23:57.501205    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85" event={"ID":"adaf9acc-e950-41ab-9807-5a0b945b407d","Type":"ContainerDied","Data":"ed53ecd492baa0000e58140f4a261071abd57c24c9e86a6716a14c3ddc43012e"}
May 28 01:24:03.167871 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:03.167819    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.33:8443: connect: connection refused"
May 28 01:24:04.053989 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:04.053940    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85" podUID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.34:8443: connect: connection refused"
May 28 01:24:04.082894 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:04.082858    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused"
May 28 01:24:04.327985 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:04.327894    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85" podUID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.34:6567/health\": dial tcp 10.133.0.34:6567: connect: connection refused"
May 28 01:24:13.167807 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:13.167758    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.33:8443: connect: connection refused"
May 28 01:24:14.054163 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:14.054112    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85" podUID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.34:8443: connect: connection refused"
May 28 01:24:14.083350 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:14.083308    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused"
May 28 01:24:14.328034 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:14.327944    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85" podUID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.34:6567/health\": dial tcp 10.133.0.34:6567: connect: connection refused"
May 28 01:24:23.168399 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:23.168342    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.33:8443: connect: connection refused"
May 28 01:24:23.168887 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:23.168522    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:24:24.053371 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:24.053319    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85" podUID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.34:8443: connect: connection refused"
May 28 01:24:24.053602 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:24.053498    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:24:24.082636 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:24.082598    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused"
May 28 01:24:24.082777 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:24.082713    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:24:24.327390 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:24.327287    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85" podUID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.34:6567/health\": dial tcp 10.133.0.34:6567: connect: connection refused"
May 28 01:24:24.327795 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:24.327440    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:24:27.261193 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.261166    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:24:27.388398 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.388338    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:24:27.392165 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.392145    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/adaf9acc-e950-41ab-9807-5a0b945b407d-offline-tls\") pod \"adaf9acc-e950-41ab-9807-5a0b945b407d\" (UID: \"adaf9acc-e950-41ab-9807-5a0b945b407d\") "
May 28 01:24:27.392269 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.392240    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/adaf9acc-e950-41ab-9807-5a0b945b407d-ui-tls\") pod \"adaf9acc-e950-41ab-9807-5a0b945b407d\" (UID: \"adaf9acc-e950-41ab-9807-5a0b945b407d\") "
May 28 01:24:27.392311 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.392271    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6wcc6\" (UniqueName: \"kubernetes.io/projected/adaf9acc-e950-41ab-9807-5a0b945b407d-kube-api-access-6wcc6\") pod \"adaf9acc-e950-41ab-9807-5a0b945b407d\" (UID: \"adaf9acc-e950-41ab-9807-5a0b945b407d\") "
May 28 01:24:27.392357 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.392308    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/adaf9acc-e950-41ab-9807-5a0b945b407d-online-tls\") pod \"adaf9acc-e950-41ab-9807-5a0b945b407d\" (UID: \"adaf9acc-e950-41ab-9807-5a0b945b407d\") "
May 28 01:24:27.392400 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.392359    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/adaf9acc-e950-41ab-9807-5a0b945b407d-registry-tls\") pod \"adaf9acc-e950-41ab-9807-5a0b945b407d\" (UID: \"adaf9acc-e950-41ab-9807-5a0b945b407d\") "
May 28 01:24:27.392463 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.392443    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/adaf9acc-e950-41ab-9807-5a0b945b407d-feast-data\") pod \"adaf9acc-e950-41ab-9807-5a0b945b407d\" (UID: \"adaf9acc-e950-41ab-9807-5a0b945b407d\") "
May 28 01:24:27.392913 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.392850    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/adaf9acc-e950-41ab-9807-5a0b945b407d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "adaf9acc-e950-41ab-9807-5a0b945b407d" (UID: "adaf9acc-e950-41ab-9807-5a0b945b407d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 01:24:27.393106 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.393078    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adaf9acc-e950-41ab-9807-5a0b945b407d-feast-data" (OuterVolumeSpecName: "feast-data") pod "adaf9acc-e950-41ab-9807-5a0b945b407d" (UID: "adaf9acc-e950-41ab-9807-5a0b945b407d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 01:24:27.394425 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.394392    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/adaf9acc-e950-41ab-9807-5a0b945b407d-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "adaf9acc-e950-41ab-9807-5a0b945b407d" (UID: "adaf9acc-e950-41ab-9807-5a0b945b407d"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:24:27.394507 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.394462    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adaf9acc-e950-41ab-9807-5a0b945b407d-kube-api-access-6wcc6" (OuterVolumeSpecName: "kube-api-access-6wcc6") pod "adaf9acc-e950-41ab-9807-5a0b945b407d" (UID: "adaf9acc-e950-41ab-9807-5a0b945b407d"). InnerVolumeSpecName "kube-api-access-6wcc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 01:24:27.394695 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.394677    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/adaf9acc-e950-41ab-9807-5a0b945b407d-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "adaf9acc-e950-41ab-9807-5a0b945b407d" (UID: "adaf9acc-e950-41ab-9807-5a0b945b407d"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:24:27.394760 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.394728    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/adaf9acc-e950-41ab-9807-5a0b945b407d-online-tls" (OuterVolumeSpecName: "online-tls") pod "adaf9acc-e950-41ab-9807-5a0b945b407d" (UID: "adaf9acc-e950-41ab-9807-5a0b945b407d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:24:27.493787 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.493747    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-feast-data\") pod \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") "
May 28 01:24:27.493787 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.493792    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-offline-tls\") pod \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") "
May 28 01:24:27.494047 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.493809    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-online-tls\") pod \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") "
May 28 01:24:27.494047 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.493848    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-registry-tls\") pod \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") "
May 28 01:24:27.494047 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.493892    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvmd7\" (UniqueName: \"kubernetes.io/projected/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-kube-api-access-wvmd7\") pod \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") "
May 28 01:24:27.494047 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.493946    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-ui-tls\") pod \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\" (UID: \"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9\") "
May 28 01:24:27.494263 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.494167    2550 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/adaf9acc-e950-41ab-9807-5a0b945b407d-offline-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:24:27.494263 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.494188    2550 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/adaf9acc-e950-41ab-9807-5a0b945b407d-ui-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:24:27.494263 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.494203    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wcc6\" (UniqueName: \"kubernetes.io/projected/adaf9acc-e950-41ab-9807-5a0b945b407d-kube-api-access-6wcc6\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:24:27.494263 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.494216    2550 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/adaf9acc-e950-41ab-9807-5a0b945b407d-online-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:24:27.494263 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.494229    2550 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/adaf9acc-e950-41ab-9807-5a0b945b407d-registry-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:24:27.494263 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.494245    2550 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/adaf9acc-e950-41ab-9807-5a0b945b407d-feast-data\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:24:27.494574 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.494374    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-feast-data" (OuterVolumeSpecName: "feast-data") pod "b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" (UID: "b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 01:24:27.496055 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.496030    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" (UID: "b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:24:27.496569 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.496541    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-online-tls" (OuterVolumeSpecName: "online-tls") pod "b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" (UID: "b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:24:27.496569 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.496558    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" (UID: "b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:24:27.496699 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.496566    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-kube-api-access-wvmd7" (OuterVolumeSpecName: "kube-api-access-wvmd7") pod "b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" (UID: "b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9"). InnerVolumeSpecName "kube-api-access-wvmd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 01:24:27.496699 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.496576    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" (UID: "b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 01:24:27.587156 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.587116    2550 generic.go:358] "Generic (PLEG): container finished" podID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerID="fb7b0f48be3aa87281e343006ec551b1b244c7c1365e437839cb4ea5540732c1" exitCode=137
May 28 01:24:27.587156 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.587147    2550 generic.go:358] "Generic (PLEG): container finished" podID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerID="2a6136880471f966868c8525b08161f83e5e1867b70dd04789da60a1530a0398" exitCode=137
May 28 01:24:27.587377 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.587186    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" event={"ID":"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9","Type":"ContainerDied","Data":"fb7b0f48be3aa87281e343006ec551b1b244c7c1365e437839cb4ea5540732c1"}
May 28 01:24:27.587377 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.587226    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" event={"ID":"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9","Type":"ContainerDied","Data":"2a6136880471f966868c8525b08161f83e5e1867b70dd04789da60a1530a0398"}
May 28 01:24:27.587377 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.587201    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"
May 28 01:24:27.587377 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.587238    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk" event={"ID":"b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9","Type":"ContainerDied","Data":"1e248eaabbbf1a1018016c3d94ffcec0c3633d1ba66cf5d43dfdf44fcc6ea8e9"}
May 28 01:24:27.587377 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.587256    2550 scope.go:117] "RemoveContainer" containerID="5f00be2a4bf5cd98cbb40592515222e4375032a3bda98ff2b6a3727fbe4d2427"
May 28 01:24:27.589630 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.589604    2550 generic.go:358] "Generic (PLEG): container finished" podID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerID="c409abca438527e6018583dc2d145548e48259b3fcb5c02d5f14f3467361b3d2" exitCode=137
May 28 01:24:27.589746 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.589688    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"
May 28 01:24:27.589814 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.589686    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85" event={"ID":"adaf9acc-e950-41ab-9807-5a0b945b407d","Type":"ContainerDied","Data":"c409abca438527e6018583dc2d145548e48259b3fcb5c02d5f14f3467361b3d2"}
May 28 01:24:27.589814 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.589788    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85" event={"ID":"adaf9acc-e950-41ab-9807-5a0b945b407d","Type":"ContainerDied","Data":"a8ac3c03f6fa65a2d4a0d9a7e8b84777b8a30fcfebbb7cd19e3cf08999ad40d6"}
May 28 01:24:27.594674 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.594649    2550 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-offline-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:24:27.594674 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.594674    2550 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-online-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:24:27.594863 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.594684    2550 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-registry-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:24:27.594863 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.594693    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvmd7\" (UniqueName: \"kubernetes.io/projected/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-kube-api-access-wvmd7\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:24:27.594863 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.594701    2550 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-ui-tls\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:24:27.594863 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.594709    2550 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9-feast-data\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:24:27.596020 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.595998    2550 scope.go:117] "RemoveContainer" containerID="fb7b0f48be3aa87281e343006ec551b1b244c7c1365e437839cb4ea5540732c1"
May 28 01:24:27.603800 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.603781    2550 scope.go:117] "RemoveContainer" containerID="2e1f569fa42ff971250cb6e22bcc9db97c12890fdd024ad187d391381acca140"
May 28 01:24:27.610878 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.610852    2550 scope.go:117] "RemoveContainer" containerID="2a6136880471f966868c8525b08161f83e5e1867b70dd04789da60a1530a0398"
May 28 01:24:27.613120 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.613097    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"]
May 28 01:24:27.618883 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.618866    2550 scope.go:117] "RemoveContainer" containerID="12f11adef9439b0b5d3f0a1e751ca58dfe1f4cff6b96131018d4e54303f357f6"
May 28 01:24:27.619530 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.619510    2550 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-v24bk"]
May 28 01:24:27.626117 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.625956    2550 scope.go:117] "RemoveContainer" containerID="db9b2dcecd98cec9ec1ecdf2a1e668389ecb085342dec6d65048ce906e84b0b7"
May 28 01:24:27.631879 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.631853    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"]
May 28 01:24:27.636270 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.636245    2550 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-c2f85"]
May 28 01:24:27.641877 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.641856    2550 scope.go:117] "RemoveContainer" containerID="5f00be2a4bf5cd98cbb40592515222e4375032a3bda98ff2b6a3727fbe4d2427"
May 28 01:24:27.642198 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:24:27.642176    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f00be2a4bf5cd98cbb40592515222e4375032a3bda98ff2b6a3727fbe4d2427\": container with ID starting with 5f00be2a4bf5cd98cbb40592515222e4375032a3bda98ff2b6a3727fbe4d2427 not found: ID does not exist" containerID="5f00be2a4bf5cd98cbb40592515222e4375032a3bda98ff2b6a3727fbe4d2427"
May 28 01:24:27.642272 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.642209    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f00be2a4bf5cd98cbb40592515222e4375032a3bda98ff2b6a3727fbe4d2427"} err="failed to get container status \"5f00be2a4bf5cd98cbb40592515222e4375032a3bda98ff2b6a3727fbe4d2427\": rpc error: code = NotFound desc = could not find container \"5f00be2a4bf5cd98cbb40592515222e4375032a3bda98ff2b6a3727fbe4d2427\": container with ID starting with 5f00be2a4bf5cd98cbb40592515222e4375032a3bda98ff2b6a3727fbe4d2427 not found: ID does not exist"
May 28 01:24:27.642272 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.642230    2550 scope.go:117] "RemoveContainer" containerID="fb7b0f48be3aa87281e343006ec551b1b244c7c1365e437839cb4ea5540732c1"
May 28 01:24:27.642502 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:24:27.642482    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb7b0f48be3aa87281e343006ec551b1b244c7c1365e437839cb4ea5540732c1\": container with ID starting with fb7b0f48be3aa87281e343006ec551b1b244c7c1365e437839cb4ea5540732c1 not found: ID does not exist" containerID="fb7b0f48be3aa87281e343006ec551b1b244c7c1365e437839cb4ea5540732c1"
May 28 01:24:27.642597 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.642511    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb7b0f48be3aa87281e343006ec551b1b244c7c1365e437839cb4ea5540732c1"} err="failed to get container status \"fb7b0f48be3aa87281e343006ec551b1b244c7c1365e437839cb4ea5540732c1\": rpc error: code = NotFound desc = could not find container \"fb7b0f48be3aa87281e343006ec551b1b244c7c1365e437839cb4ea5540732c1\": container with ID starting with fb7b0f48be3aa87281e343006ec551b1b244c7c1365e437839cb4ea5540732c1 not found: ID does not exist"
May 28 01:24:27.642597 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.642531    2550 scope.go:117] "RemoveContainer" containerID="2e1f569fa42ff971250cb6e22bcc9db97c12890fdd024ad187d391381acca140"
May 28 01:24:27.642804 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:24:27.642788    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e1f569fa42ff971250cb6e22bcc9db97c12890fdd024ad187d391381acca140\": container with ID starting with 2e1f569fa42ff971250cb6e22bcc9db97c12890fdd024ad187d391381acca140 not found: ID does not exist" containerID="2e1f569fa42ff971250cb6e22bcc9db97c12890fdd024ad187d391381acca140"
May 28 01:24:27.642845 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.642808    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e1f569fa42ff971250cb6e22bcc9db97c12890fdd024ad187d391381acca140"} err="failed to get container status \"2e1f569fa42ff971250cb6e22bcc9db97c12890fdd024ad187d391381acca140\": rpc error: code = NotFound desc = could not find container \"2e1f569fa42ff971250cb6e22bcc9db97c12890fdd024ad187d391381acca140\": container with ID starting with 2e1f569fa42ff971250cb6e22bcc9db97c12890fdd024ad187d391381acca140 not found: ID does not exist"
May 28 01:24:27.642845 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.642822    2550 scope.go:117] "RemoveContainer" containerID="2a6136880471f966868c8525b08161f83e5e1867b70dd04789da60a1530a0398"
May 28 01:24:27.643052 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:24:27.643035    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a6136880471f966868c8525b08161f83e5e1867b70dd04789da60a1530a0398\": container with ID starting with 2a6136880471f966868c8525b08161f83e5e1867b70dd04789da60a1530a0398 not found: ID does not exist" containerID="2a6136880471f966868c8525b08161f83e5e1867b70dd04789da60a1530a0398"
May 28 01:24:27.643108 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.643056    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a6136880471f966868c8525b08161f83e5e1867b70dd04789da60a1530a0398"} err="failed to get container status \"2a6136880471f966868c8525b08161f83e5e1867b70dd04789da60a1530a0398\": rpc error: code = NotFound desc = could not find container \"2a6136880471f966868c8525b08161f83e5e1867b70dd04789da60a1530a0398\": container with ID starting with 2a6136880471f966868c8525b08161f83e5e1867b70dd04789da60a1530a0398 not found: ID does not exist"
May 28 01:24:27.643108 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.643098    2550 scope.go:117] "RemoveContainer" containerID="12f11adef9439b0b5d3f0a1e751ca58dfe1f4cff6b96131018d4e54303f357f6"
May 28 01:24:27.643312 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:24:27.643299    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12f11adef9439b0b5d3f0a1e751ca58dfe1f4cff6b96131018d4e54303f357f6\": container with ID starting with 12f11adef9439b0b5d3f0a1e751ca58dfe1f4cff6b96131018d4e54303f357f6 not found: ID does not exist" containerID="12f11adef9439b0b5d3f0a1e751ca58dfe1f4cff6b96131018d4e54303f357f6"
May 28 01:24:27.643350 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.643315    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12f11adef9439b0b5d3f0a1e751ca58dfe1f4cff6b96131018d4e54303f357f6"} err="failed to get container status \"12f11adef9439b0b5d3f0a1e751ca58dfe1f4cff6b96131018d4e54303f357f6\": rpc error: code = NotFound desc = could not find container \"12f11adef9439b0b5d3f0a1e751ca58dfe1f4cff6b96131018d4e54303f357f6\": container with ID starting with 12f11adef9439b0b5d3f0a1e751ca58dfe1f4cff6b96131018d4e54303f357f6 not found: ID does not exist"
May 28 01:24:27.643350 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.643326    2550 scope.go:117] "RemoveContainer" containerID="db9b2dcecd98cec9ec1ecdf2a1e668389ecb085342dec6d65048ce906e84b0b7"
May 28 01:24:27.643544 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:24:27.643529    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db9b2dcecd98cec9ec1ecdf2a1e668389ecb085342dec6d65048ce906e84b0b7\": container with ID starting with db9b2dcecd98cec9ec1ecdf2a1e668389ecb085342dec6d65048ce906e84b0b7 not found: ID does not exist" containerID="db9b2dcecd98cec9ec1ecdf2a1e668389ecb085342dec6d65048ce906e84b0b7"
May 28 01:24:27.643596 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.643548    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db9b2dcecd98cec9ec1ecdf2a1e668389ecb085342dec6d65048ce906e84b0b7"} err="failed to get container status \"db9b2dcecd98cec9ec1ecdf2a1e668389ecb085342dec6d65048ce906e84b0b7\": rpc error: code = NotFound desc = could not find container \"db9b2dcecd98cec9ec1ecdf2a1e668389ecb085342dec6d65048ce906e84b0b7\": container with ID starting with db9b2dcecd98cec9ec1ecdf2a1e668389ecb085342dec6d65048ce906e84b0b7 not found: ID does not exist"
May 28 01:24:27.643596 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.643560    2550 scope.go:117] "RemoveContainer" containerID="5f00be2a4bf5cd98cbb40592515222e4375032a3bda98ff2b6a3727fbe4d2427"
May 28 01:24:27.643758 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.643741    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f00be2a4bf5cd98cbb40592515222e4375032a3bda98ff2b6a3727fbe4d2427"} err="failed to get container status \"5f00be2a4bf5cd98cbb40592515222e4375032a3bda98ff2b6a3727fbe4d2427\": rpc error: code = NotFound desc = could not find container \"5f00be2a4bf5cd98cbb40592515222e4375032a3bda98ff2b6a3727fbe4d2427\": container with ID starting with 5f00be2a4bf5cd98cbb40592515222e4375032a3bda98ff2b6a3727fbe4d2427 not found: ID does not exist"
May 28 01:24:27.643801 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.643758    2550 scope.go:117] "RemoveContainer" containerID="fb7b0f48be3aa87281e343006ec551b1b244c7c1365e437839cb4ea5540732c1"
May 28 01:24:27.643974 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.643956    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb7b0f48be3aa87281e343006ec551b1b244c7c1365e437839cb4ea5540732c1"} err="failed to get container status \"fb7b0f48be3aa87281e343006ec551b1b244c7c1365e437839cb4ea5540732c1\": rpc error: code = NotFound desc = could not find container \"fb7b0f48be3aa87281e343006ec551b1b244c7c1365e437839cb4ea5540732c1\": container with ID starting with fb7b0f48be3aa87281e343006ec551b1b244c7c1365e437839cb4ea5540732c1 not found: ID does not exist"
May 28 01:24:27.644032 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.643975    2550 scope.go:117] "RemoveContainer" containerID="2e1f569fa42ff971250cb6e22bcc9db97c12890fdd024ad187d391381acca140"
May 28 01:24:27.644172 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.644156    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e1f569fa42ff971250cb6e22bcc9db97c12890fdd024ad187d391381acca140"} err="failed to get container status \"2e1f569fa42ff971250cb6e22bcc9db97c12890fdd024ad187d391381acca140\": rpc error: code = NotFound desc = could not find container \"2e1f569fa42ff971250cb6e22bcc9db97c12890fdd024ad187d391381acca140\": container with ID starting with 2e1f569fa42ff971250cb6e22bcc9db97c12890fdd024ad187d391381acca140 not found: ID does not exist"
May 28 01:24:27.644214 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.644173    2550 scope.go:117] "RemoveContainer" containerID="2a6136880471f966868c8525b08161f83e5e1867b70dd04789da60a1530a0398"
May 28 01:24:27.644353 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.644339    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a6136880471f966868c8525b08161f83e5e1867b70dd04789da60a1530a0398"} err="failed to get container status \"2a6136880471f966868c8525b08161f83e5e1867b70dd04789da60a1530a0398\": rpc error: code = NotFound desc = could not find container \"2a6136880471f966868c8525b08161f83e5e1867b70dd04789da60a1530a0398\": container with ID starting with 2a6136880471f966868c8525b08161f83e5e1867b70dd04789da60a1530a0398 not found: ID does not exist"
May 28 01:24:27.644405 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.644354    2550 scope.go:117] "RemoveContainer" containerID="12f11adef9439b0b5d3f0a1e751ca58dfe1f4cff6b96131018d4e54303f357f6"
May 28 01:24:27.644542 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.644526    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12f11adef9439b0b5d3f0a1e751ca58dfe1f4cff6b96131018d4e54303f357f6"} err="failed to get container status \"12f11adef9439b0b5d3f0a1e751ca58dfe1f4cff6b96131018d4e54303f357f6\": rpc error: code = NotFound desc = could not find container \"12f11adef9439b0b5d3f0a1e751ca58dfe1f4cff6b96131018d4e54303f357f6\": container with ID starting with 12f11adef9439b0b5d3f0a1e751ca58dfe1f4cff6b96131018d4e54303f357f6 not found: ID does not exist"
May 28 01:24:27.644593 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.644543    2550 scope.go:117] "RemoveContainer" containerID="db9b2dcecd98cec9ec1ecdf2a1e668389ecb085342dec6d65048ce906e84b0b7"
May 28 01:24:27.644723 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.644702    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db9b2dcecd98cec9ec1ecdf2a1e668389ecb085342dec6d65048ce906e84b0b7"} err="failed to get container status \"db9b2dcecd98cec9ec1ecdf2a1e668389ecb085342dec6d65048ce906e84b0b7\": rpc error: code = NotFound desc = could not find container \"db9b2dcecd98cec9ec1ecdf2a1e668389ecb085342dec6d65048ce906e84b0b7\": container with ID starting with db9b2dcecd98cec9ec1ecdf2a1e668389ecb085342dec6d65048ce906e84b0b7 not found: ID does not exist"
May 28 01:24:27.644771 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.644722    2550 scope.go:117] "RemoveContainer" containerID="35ca1687df3417aa0cbf8241202757629269c5eb11ffd687846da029ed2cd5ec"
May 28 01:24:27.652368 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.652348    2550 scope.go:117] "RemoveContainer" containerID="c409abca438527e6018583dc2d145548e48259b3fcb5c02d5f14f3467361b3d2"
May 28 01:24:27.659240 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.659219    2550 scope.go:117] "RemoveContainer" containerID="ed53ecd492baa0000e58140f4a261071abd57c24c9e86a6716a14c3ddc43012e"
May 28 01:24:27.666688 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.666667    2550 scope.go:117] "RemoveContainer" containerID="703f2bfee918a2cf3c60bb6a2847155df56b164c1dced8d32dd2dd8656670ee5"
May 28 01:24:27.673592 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.673570    2550 scope.go:117] "RemoveContainer" containerID="5765c903ceebe83e9f95d7ba3471aee416e13c9a9f2f70cd443a43eacda17911"
May 28 01:24:27.680634 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.680616    2550 scope.go:117] "RemoveContainer" containerID="35ca1687df3417aa0cbf8241202757629269c5eb11ffd687846da029ed2cd5ec"
May 28 01:24:27.680889 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:24:27.680871    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35ca1687df3417aa0cbf8241202757629269c5eb11ffd687846da029ed2cd5ec\": container with ID starting with 35ca1687df3417aa0cbf8241202757629269c5eb11ffd687846da029ed2cd5ec not found: ID does not exist" containerID="35ca1687df3417aa0cbf8241202757629269c5eb11ffd687846da029ed2cd5ec"
May 28 01:24:27.680941 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.680897    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35ca1687df3417aa0cbf8241202757629269c5eb11ffd687846da029ed2cd5ec"} err="failed to get container status \"35ca1687df3417aa0cbf8241202757629269c5eb11ffd687846da029ed2cd5ec\": rpc error: code = NotFound desc = could not find container \"35ca1687df3417aa0cbf8241202757629269c5eb11ffd687846da029ed2cd5ec\": container with ID starting with 35ca1687df3417aa0cbf8241202757629269c5eb11ffd687846da029ed2cd5ec not found: ID does not exist"
May 28 01:24:27.680941 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.680915    2550 scope.go:117] "RemoveContainer" containerID="c409abca438527e6018583dc2d145548e48259b3fcb5c02d5f14f3467361b3d2"
May 28 01:24:27.681174 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:24:27.681152    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c409abca438527e6018583dc2d145548e48259b3fcb5c02d5f14f3467361b3d2\": container with ID starting with c409abca438527e6018583dc2d145548e48259b3fcb5c02d5f14f3467361b3d2 not found: ID does not exist" containerID="c409abca438527e6018583dc2d145548e48259b3fcb5c02d5f14f3467361b3d2"
May 28 01:24:27.681217 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.681182    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c409abca438527e6018583dc2d145548e48259b3fcb5c02d5f14f3467361b3d2"} err="failed to get container status \"c409abca438527e6018583dc2d145548e48259b3fcb5c02d5f14f3467361b3d2\": rpc error: code = NotFound desc = could not find container \"c409abca438527e6018583dc2d145548e48259b3fcb5c02d5f14f3467361b3d2\": container with ID starting with c409abca438527e6018583dc2d145548e48259b3fcb5c02d5f14f3467361b3d2 not found: ID does not exist"
May 28 01:24:27.681217 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.681205    2550 scope.go:117] "RemoveContainer" containerID="ed53ecd492baa0000e58140f4a261071abd57c24c9e86a6716a14c3ddc43012e"
May 28 01:24:27.681475 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:24:27.681454    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed53ecd492baa0000e58140f4a261071abd57c24c9e86a6716a14c3ddc43012e\": container with ID starting with ed53ecd492baa0000e58140f4a261071abd57c24c9e86a6716a14c3ddc43012e not found: ID does not exist" containerID="ed53ecd492baa0000e58140f4a261071abd57c24c9e86a6716a14c3ddc43012e"
May 28 01:24:27.681544 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.681483    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed53ecd492baa0000e58140f4a261071abd57c24c9e86a6716a14c3ddc43012e"} err="failed to get container status \"ed53ecd492baa0000e58140f4a261071abd57c24c9e86a6716a14c3ddc43012e\": rpc error: code = NotFound desc = could not find container \"ed53ecd492baa0000e58140f4a261071abd57c24c9e86a6716a14c3ddc43012e\": container with ID starting with ed53ecd492baa0000e58140f4a261071abd57c24c9e86a6716a14c3ddc43012e not found: ID does not exist"
May 28 01:24:27.681544 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.681499    2550 scope.go:117] "RemoveContainer" containerID="703f2bfee918a2cf3c60bb6a2847155df56b164c1dced8d32dd2dd8656670ee5"
May 28 01:24:27.681761 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:24:27.681741    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"703f2bfee918a2cf3c60bb6a2847155df56b164c1dced8d32dd2dd8656670ee5\": container with ID starting with 703f2bfee918a2cf3c60bb6a2847155df56b164c1dced8d32dd2dd8656670ee5 not found: ID does not exist" containerID="703f2bfee918a2cf3c60bb6a2847155df56b164c1dced8d32dd2dd8656670ee5"
May 28 01:24:27.681830 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.681767    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"703f2bfee918a2cf3c60bb6a2847155df56b164c1dced8d32dd2dd8656670ee5"} err="failed to get container status \"703f2bfee918a2cf3c60bb6a2847155df56b164c1dced8d32dd2dd8656670ee5\": rpc error: code = NotFound desc = could not find container \"703f2bfee918a2cf3c60bb6a2847155df56b164c1dced8d32dd2dd8656670ee5\": container with ID starting with 703f2bfee918a2cf3c60bb6a2847155df56b164c1dced8d32dd2dd8656670ee5 not found: ID does not exist"
May 28 01:24:27.681830 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.681783    2550 scope.go:117] "RemoveContainer" containerID="5765c903ceebe83e9f95d7ba3471aee416e13c9a9f2f70cd443a43eacda17911"
May 28 01:24:27.682038 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:24:27.682024    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5765c903ceebe83e9f95d7ba3471aee416e13c9a9f2f70cd443a43eacda17911\": container with ID starting with 5765c903ceebe83e9f95d7ba3471aee416e13c9a9f2f70cd443a43eacda17911 not found: ID does not exist" containerID="5765c903ceebe83e9f95d7ba3471aee416e13c9a9f2f70cd443a43eacda17911"
May 28 01:24:27.682086 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:27.682040    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5765c903ceebe83e9f95d7ba3471aee416e13c9a9f2f70cd443a43eacda17911"} err="failed to get container status \"5765c903ceebe83e9f95d7ba3471aee416e13c9a9f2f70cd443a43eacda17911\": rpc error: code = NotFound desc = could not find container \"5765c903ceebe83e9f95d7ba3471aee416e13c9a9f2f70cd443a43eacda17911\": container with ID starting with 5765c903ceebe83e9f95d7ba3471aee416e13c9a9f2f70cd443a43eacda17911 not found: ID does not exist"
May 28 01:24:29.323920 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:29.323885    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adaf9acc-e950-41ab-9807-5a0b945b407d" path="/var/lib/kubelet/pods/adaf9acc-e950-41ab-9807-5a0b945b407d/volumes"
May 28 01:24:29.324410 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:29.324397    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" path="/var/lib/kubelet/pods/b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9/volumes"
May 28 01:24:45.637027 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.636991    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4q6f4/must-gather-gknx5"]
May 28 01:24:45.637394 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637236    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="registry"
May 28 01:24:45.637394 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637246    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="registry"
May 28 01:24:45.637394 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637257    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="feast-apply"
May 28 01:24:45.637394 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637263    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="feast-apply"
May 28 01:24:45.637394 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637269    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="offline"
May 28 01:24:45.637394 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637275    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="offline"
May 28 01:24:45.637394 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637281    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerName="ui"
May 28 01:24:45.637394 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637287    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerName="ui"
May 28 01:24:45.637394 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637297    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerName="feast-init"
May 28 01:24:45.637394 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637302    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerName="feast-init"
May 28 01:24:45.637394 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637309    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="feast-init"
May 28 01:24:45.637394 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637314    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="feast-init"
May 28 01:24:45.637394 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637320    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerName="feast-apply"
May 28 01:24:45.637394 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637325    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerName="feast-apply"
May 28 01:24:45.637394 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637332    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerName="online"
May 28 01:24:45.637394 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637337    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerName="online"
May 28 01:24:45.637394 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637343    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerName="offline"
May 28 01:24:45.637394 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637348    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerName="offline"
May 28 01:24:45.637394 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637355    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="ui"
May 28 01:24:45.637394 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637359    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="ui"
May 28 01:24:45.637394 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637365    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="online"
May 28 01:24:45.637394 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637370    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="online"
May 28 01:24:45.638094 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637407    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="offline"
May 28 01:24:45.638094 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637429    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerName="online"
May 28 01:24:45.638094 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637445    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="ui"
May 28 01:24:45.638094 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637451    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="registry"
May 28 01:24:45.638094 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637457    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerName="offline"
May 28 01:24:45.638094 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637462    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="adaf9acc-e950-41ab-9807-5a0b945b407d" containerName="ui"
May 28 01:24:45.638094 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.637468    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="b7c2b0b3-9ea2-4129-bdce-8e7d69d215b9" containerName="online"
May 28 01:24:45.641873 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.641853    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4q6f4/must-gather-gknx5"
May 28 01:24:45.643912 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.643890    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4q6f4\"/\"openshift-service-ca.crt\""
May 28 01:24:45.644533 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.644511    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4q6f4\"/\"default-dockercfg-45wrg\""
May 28 01:24:45.644648 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.644551    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4q6f4\"/\"kube-root-ca.crt\""
May 28 01:24:45.647220 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.647198    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4q6f4/must-gather-gknx5"]
May 28 01:24:45.739673 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.739642    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/404c3b8d-81fd-4aad-9841-3441cbb2d3f8-must-gather-output\") pod \"must-gather-gknx5\" (UID: \"404c3b8d-81fd-4aad-9841-3441cbb2d3f8\") " pod="openshift-must-gather-4q6f4/must-gather-gknx5"
May 28 01:24:45.739841 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.739690    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwgxj\" (UniqueName: \"kubernetes.io/projected/404c3b8d-81fd-4aad-9841-3441cbb2d3f8-kube-api-access-xwgxj\") pod \"must-gather-gknx5\" (UID: \"404c3b8d-81fd-4aad-9841-3441cbb2d3f8\") " pod="openshift-must-gather-4q6f4/must-gather-gknx5"
May 28 01:24:45.840493 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.840462    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/404c3b8d-81fd-4aad-9841-3441cbb2d3f8-must-gather-output\") pod \"must-gather-gknx5\" (UID: \"404c3b8d-81fd-4aad-9841-3441cbb2d3f8\") " pod="openshift-must-gather-4q6f4/must-gather-gknx5"
May 28 01:24:45.840493 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.840498    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xwgxj\" (UniqueName: \"kubernetes.io/projected/404c3b8d-81fd-4aad-9841-3441cbb2d3f8-kube-api-access-xwgxj\") pod \"must-gather-gknx5\" (UID: \"404c3b8d-81fd-4aad-9841-3441cbb2d3f8\") " pod="openshift-must-gather-4q6f4/must-gather-gknx5"
May 28 01:24:45.840803 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.840784    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/404c3b8d-81fd-4aad-9841-3441cbb2d3f8-must-gather-output\") pod \"must-gather-gknx5\" (UID: \"404c3b8d-81fd-4aad-9841-3441cbb2d3f8\") " pod="openshift-must-gather-4q6f4/must-gather-gknx5"
May 28 01:24:45.849960 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.849931    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwgxj\" (UniqueName: \"kubernetes.io/projected/404c3b8d-81fd-4aad-9841-3441cbb2d3f8-kube-api-access-xwgxj\") pod \"must-gather-gknx5\" (UID: \"404c3b8d-81fd-4aad-9841-3441cbb2d3f8\") " pod="openshift-must-gather-4q6f4/must-gather-gknx5"
May 28 01:24:45.965174 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:45.965136    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4q6f4/must-gather-gknx5"
May 28 01:24:46.088343 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:46.088317    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4q6f4/must-gather-gknx5"]
May 28 01:24:46.090855 ip-10-0-131-73 kubenswrapper[2550]: W0528 01:24:46.090823    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod404c3b8d_81fd_4aad_9841_3441cbb2d3f8.slice/crio-bf4f757e7fb51b5377cfd92f6f7fc25c34de1e710632e84797154e087959c3c5 WatchSource:0}: Error finding container bf4f757e7fb51b5377cfd92f6f7fc25c34de1e710632e84797154e087959c3c5: Status 404 returned error can't find the container with id bf4f757e7fb51b5377cfd92f6f7fc25c34de1e710632e84797154e087959c3c5
May 28 01:24:46.644901 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:46.644866    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4q6f4/must-gather-gknx5" event={"ID":"404c3b8d-81fd-4aad-9841-3441cbb2d3f8","Type":"ContainerStarted","Data":"bf4f757e7fb51b5377cfd92f6f7fc25c34de1e710632e84797154e087959c3c5"}
May 28 01:24:52.666777 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:52.666738    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4q6f4/must-gather-gknx5" event={"ID":"404c3b8d-81fd-4aad-9841-3441cbb2d3f8","Type":"ContainerStarted","Data":"e56888f8373b220a486c1beefc2a0c4aa167e2c63599afb33f896a5dc49e4528"}
May 28 01:24:52.666777 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:52.666781    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4q6f4/must-gather-gknx5" event={"ID":"404c3b8d-81fd-4aad-9841-3441cbb2d3f8","Type":"ContainerStarted","Data":"25b79021b94bf07279be773eb224575be8984abe0bd57265ff150bd5b6fb52fb"}
May 28 01:24:52.685122 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:24:52.685058    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-4q6f4/must-gather-gknx5" podStartSLOduration=1.475875732 podStartE2EDuration="7.685038059s" podCreationTimestamp="2026-05-28 01:24:45 +0000 UTC" firstStartedPulling="2026-05-28 01:24:46.092588009 +0000 UTC m=+1745.373545080" lastFinishedPulling="2026-05-28 01:24:52.301750328 +0000 UTC m=+1751.582707407" observedRunningTime="2026-05-28 01:24:52.68280702 +0000 UTC m=+1751.963764112" watchObservedRunningTime="2026-05-28 01:24:52.685038059 +0000 UTC m=+1751.965995156"
May 28 01:25:02.695258 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:02.695201    2550 generic.go:358] "Generic (PLEG): container finished" podID="404c3b8d-81fd-4aad-9841-3441cbb2d3f8" containerID="25b79021b94bf07279be773eb224575be8984abe0bd57265ff150bd5b6fb52fb" exitCode=0
May 28 01:25:02.695767 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:02.695274    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4q6f4/must-gather-gknx5" event={"ID":"404c3b8d-81fd-4aad-9841-3441cbb2d3f8","Type":"ContainerDied","Data":"25b79021b94bf07279be773eb224575be8984abe0bd57265ff150bd5b6fb52fb"}
May 28 01:25:02.695767 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:02.695619    2550 scope.go:117] "RemoveContainer" containerID="25b79021b94bf07279be773eb224575be8984abe0bd57265ff150bd5b6fb52fb"
May 28 01:25:03.033516 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:03.033431    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-4q6f4_must-gather-gknx5_404c3b8d-81fd-4aad-9841-3441cbb2d3f8/gather/0.log"
May 28 01:25:03.514599 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:03.514565    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-m2w8l/must-gather-zh74m"]
May 28 01:25:03.517599 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:03.517581    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m2w8l/must-gather-zh74m"
May 28 01:25:03.519660 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:03.519637    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-m2w8l\"/\"openshift-service-ca.crt\""
May 28 01:25:03.519792 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:03.519638    2550 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-m2w8l\"/\"kube-root-ca.crt\""
May 28 01:25:03.520097 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:03.520079    2550 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-m2w8l\"/\"default-dockercfg-hc6tl\""
May 28 01:25:03.524156 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:03.524130    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m2w8l/must-gather-zh74m"]
May 28 01:25:03.594687 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:03.594648    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rplxz\" (UniqueName: \"kubernetes.io/projected/ae63045b-2e5d-4e3a-8578-9938ed70392e-kube-api-access-rplxz\") pod \"must-gather-zh74m\" (UID: \"ae63045b-2e5d-4e3a-8578-9938ed70392e\") " pod="openshift-must-gather-m2w8l/must-gather-zh74m"
May 28 01:25:03.594888 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:03.594706    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ae63045b-2e5d-4e3a-8578-9938ed70392e-must-gather-output\") pod \"must-gather-zh74m\" (UID: \"ae63045b-2e5d-4e3a-8578-9938ed70392e\") " pod="openshift-must-gather-m2w8l/must-gather-zh74m"
May 28 01:25:03.695609 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:03.695557    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ae63045b-2e5d-4e3a-8578-9938ed70392e-must-gather-output\") pod \"must-gather-zh74m\" (UID: \"ae63045b-2e5d-4e3a-8578-9938ed70392e\") " pod="openshift-must-gather-m2w8l/must-gather-zh74m"
May 28 01:25:03.696102 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:03.695651    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rplxz\" (UniqueName: \"kubernetes.io/projected/ae63045b-2e5d-4e3a-8578-9938ed70392e-kube-api-access-rplxz\") pod \"must-gather-zh74m\" (UID: \"ae63045b-2e5d-4e3a-8578-9938ed70392e\") " pod="openshift-must-gather-m2w8l/must-gather-zh74m"
May 28 01:25:03.696102 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:03.695886    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ae63045b-2e5d-4e3a-8578-9938ed70392e-must-gather-output\") pod \"must-gather-zh74m\" (UID: \"ae63045b-2e5d-4e3a-8578-9938ed70392e\") " pod="openshift-must-gather-m2w8l/must-gather-zh74m"
May 28 01:25:03.703329 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:03.703305    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rplxz\" (UniqueName: \"kubernetes.io/projected/ae63045b-2e5d-4e3a-8578-9938ed70392e-kube-api-access-rplxz\") pod \"must-gather-zh74m\" (UID: \"ae63045b-2e5d-4e3a-8578-9938ed70392e\") " pod="openshift-must-gather-m2w8l/must-gather-zh74m"
May 28 01:25:03.827677 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:03.827570    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m2w8l/must-gather-zh74m"
May 28 01:25:03.953386 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:03.953347    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m2w8l/must-gather-zh74m"]
May 28 01:25:03.956425 ip-10-0-131-73 kubenswrapper[2550]: W0528 01:25:03.956380    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae63045b_2e5d_4e3a_8578_9938ed70392e.slice/crio-2349525ac4901c9e07df02b7e8b2cab36b87464a5873c251210fb4b3bf9cf1ff WatchSource:0}: Error finding container 2349525ac4901c9e07df02b7e8b2cab36b87464a5873c251210fb4b3bf9cf1ff: Status 404 returned error can't find the container with id 2349525ac4901c9e07df02b7e8b2cab36b87464a5873c251210fb4b3bf9cf1ff
May 28 01:25:03.958157 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:03.958140    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 01:25:04.702975 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:04.702939    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m2w8l/must-gather-zh74m" event={"ID":"ae63045b-2e5d-4e3a-8578-9938ed70392e","Type":"ContainerStarted","Data":"2349525ac4901c9e07df02b7e8b2cab36b87464a5873c251210fb4b3bf9cf1ff"}
May 28 01:25:05.708953 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:05.708914    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m2w8l/must-gather-zh74m" event={"ID":"ae63045b-2e5d-4e3a-8578-9938ed70392e","Type":"ContainerStarted","Data":"b053afe30081e98f78bdae3a0df4e2d94ca4cf58915eb114372da7a81ddc3f98"}
May 28 01:25:05.708953 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:05.708957    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m2w8l/must-gather-zh74m" event={"ID":"ae63045b-2e5d-4e3a-8578-9938ed70392e","Type":"ContainerStarted","Data":"64872864426ef374424c9f58b749b01f21aca66ebf94823bb66163534cac0310"}
May 28 01:25:05.725354 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:05.725278    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-m2w8l/must-gather-zh74m" podStartSLOduration=1.953094116 podStartE2EDuration="2.725257058s" podCreationTimestamp="2026-05-28 01:25:03 +0000 UTC" firstStartedPulling="2026-05-28 01:25:03.958293456 +0000 UTC m=+1763.239250527" lastFinishedPulling="2026-05-28 01:25:04.730456397 +0000 UTC m=+1764.011413469" observedRunningTime="2026-05-28 01:25:05.723694096 +0000 UTC m=+1765.004651195" watchObservedRunningTime="2026-05-28 01:25:05.725257058 +0000 UTC m=+1765.006214152"
May 28 01:25:06.148735 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:06.148621    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qw4tp_42201706-6a2e-412d-8072-15c3476baba9/global-pull-secret-syncer/0.log"
May 28 01:25:06.226023 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:06.225995    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lhxvp_1f856cae-503d-46e6-83f2-ef3127046b5c/konnectivity-agent/0.log"
May 28 01:25:06.293111 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:06.293079    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-73.ec2.internal_4c51bc3462f829dd646da56091097a15/haproxy/0.log"
May 28 01:25:08.351101 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:08.351063    2550 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-4q6f4/must-gather-gknx5"]
May 28 01:25:08.352072 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:08.352029    2550 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-must-gather-4q6f4/must-gather-gknx5" podUID="404c3b8d-81fd-4aad-9841-3441cbb2d3f8" containerName="copy" containerID="cri-o://e56888f8373b220a486c1beefc2a0c4aa167e2c63599afb33f896a5dc49e4528" gracePeriod=2
May 28 01:25:08.354773 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:08.354729    2550 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-4q6f4/must-gather-gknx5"]
May 28 01:25:08.356499 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:08.356462    2550 status_manager.go:895] "Failed to get status for pod" podUID="404c3b8d-81fd-4aad-9841-3441cbb2d3f8" pod="openshift-must-gather-4q6f4/must-gather-gknx5" err="pods \"must-gather-gknx5\" is forbidden: User \"system:node:ip-10-0-131-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-4q6f4\": no relationship found between node 'ip-10-0-131-73.ec2.internal' and this object"
May 28 01:25:08.688622 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:08.685701    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-4q6f4_must-gather-gknx5_404c3b8d-81fd-4aad-9841-3441cbb2d3f8/copy/0.log"
May 28 01:25:08.688622 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:08.686109    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4q6f4/must-gather-gknx5"
May 28 01:25:08.691859 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:08.689476    2550 status_manager.go:895] "Failed to get status for pod" podUID="404c3b8d-81fd-4aad-9841-3441cbb2d3f8" pod="openshift-must-gather-4q6f4/must-gather-gknx5" err="pods \"must-gather-gknx5\" is forbidden: User \"system:node:ip-10-0-131-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-4q6f4\": no relationship found between node 'ip-10-0-131-73.ec2.internal' and this object"
May 28 01:25:08.730734 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:08.730642    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-4q6f4_must-gather-gknx5_404c3b8d-81fd-4aad-9841-3441cbb2d3f8/copy/0.log"
May 28 01:25:08.736230 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:08.735590    2550 generic.go:358] "Generic (PLEG): container finished" podID="404c3b8d-81fd-4aad-9841-3441cbb2d3f8" containerID="e56888f8373b220a486c1beefc2a0c4aa167e2c63599afb33f896a5dc49e4528" exitCode=143
May 28 01:25:08.736230 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:08.735719    2550 scope.go:117] "RemoveContainer" containerID="e56888f8373b220a486c1beefc2a0c4aa167e2c63599afb33f896a5dc49e4528"
May 28 01:25:08.736230 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:08.735864    2550 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4q6f4/must-gather-gknx5"
May 28 01:25:08.737695 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:08.737641    2550 status_manager.go:895] "Failed to get status for pod" podUID="404c3b8d-81fd-4aad-9841-3441cbb2d3f8" pod="openshift-must-gather-4q6f4/must-gather-gknx5" err="pods \"must-gather-gknx5\" is forbidden: User \"system:node:ip-10-0-131-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-4q6f4\": no relationship found between node 'ip-10-0-131-73.ec2.internal' and this object"
May 28 01:25:08.746994 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:08.746965    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwgxj\" (UniqueName: \"kubernetes.io/projected/404c3b8d-81fd-4aad-9841-3441cbb2d3f8-kube-api-access-xwgxj\") pod \"404c3b8d-81fd-4aad-9841-3441cbb2d3f8\" (UID: \"404c3b8d-81fd-4aad-9841-3441cbb2d3f8\") "
May 28 01:25:08.747127 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:08.747016    2550 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/404c3b8d-81fd-4aad-9841-3441cbb2d3f8-must-gather-output\") pod \"404c3b8d-81fd-4aad-9841-3441cbb2d3f8\" (UID: \"404c3b8d-81fd-4aad-9841-3441cbb2d3f8\") "
May 28 01:25:08.747612 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:08.747567    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/404c3b8d-81fd-4aad-9841-3441cbb2d3f8-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "404c3b8d-81fd-4aad-9841-3441cbb2d3f8" (UID: "404c3b8d-81fd-4aad-9841-3441cbb2d3f8"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 01:25:08.757777 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:08.756362    2550 scope.go:117] "RemoveContainer" containerID="25b79021b94bf07279be773eb224575be8984abe0bd57265ff150bd5b6fb52fb"
May 28 01:25:08.759931 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:08.759882    2550 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/404c3b8d-81fd-4aad-9841-3441cbb2d3f8-kube-api-access-xwgxj" (OuterVolumeSpecName: "kube-api-access-xwgxj") pod "404c3b8d-81fd-4aad-9841-3441cbb2d3f8" (UID: "404c3b8d-81fd-4aad-9841-3441cbb2d3f8"). InnerVolumeSpecName "kube-api-access-xwgxj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 01:25:08.785617 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:08.785579    2550 scope.go:117] "RemoveContainer" containerID="e56888f8373b220a486c1beefc2a0c4aa167e2c63599afb33f896a5dc49e4528"
May 28 01:25:08.786010 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:25:08.785961    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e56888f8373b220a486c1beefc2a0c4aa167e2c63599afb33f896a5dc49e4528\": container with ID starting with e56888f8373b220a486c1beefc2a0c4aa167e2c63599afb33f896a5dc49e4528 not found: ID does not exist" containerID="e56888f8373b220a486c1beefc2a0c4aa167e2c63599afb33f896a5dc49e4528"
May 28 01:25:08.786110 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:08.785999    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e56888f8373b220a486c1beefc2a0c4aa167e2c63599afb33f896a5dc49e4528"} err="failed to get container status \"e56888f8373b220a486c1beefc2a0c4aa167e2c63599afb33f896a5dc49e4528\": rpc error: code = NotFound desc = could not find container \"e56888f8373b220a486c1beefc2a0c4aa167e2c63599afb33f896a5dc49e4528\": container with ID starting with e56888f8373b220a486c1beefc2a0c4aa167e2c63599afb33f896a5dc49e4528 not found: ID does not exist"
May 28 01:25:08.786110 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:08.786038    2550 scope.go:117] "RemoveContainer" containerID="25b79021b94bf07279be773eb224575be8984abe0bd57265ff150bd5b6fb52fb"
May 28 01:25:08.786372 ip-10-0-131-73 kubenswrapper[2550]: E0528 01:25:08.786305    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25b79021b94bf07279be773eb224575be8984abe0bd57265ff150bd5b6fb52fb\": container with ID starting with 25b79021b94bf07279be773eb224575be8984abe0bd57265ff150bd5b6fb52fb not found: ID does not exist" containerID="25b79021b94bf07279be773eb224575be8984abe0bd57265ff150bd5b6fb52fb"
May 28 01:25:08.786372 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:08.786338    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25b79021b94bf07279be773eb224575be8984abe0bd57265ff150bd5b6fb52fb"} err="failed to get container status \"25b79021b94bf07279be773eb224575be8984abe0bd57265ff150bd5b6fb52fb\": rpc error: code = NotFound desc = could not find container \"25b79021b94bf07279be773eb224575be8984abe0bd57265ff150bd5b6fb52fb\": container with ID starting with 25b79021b94bf07279be773eb224575be8984abe0bd57265ff150bd5b6fb52fb not found: ID does not exist"
May 28 01:25:08.847863 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:08.847789    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwgxj\" (UniqueName: \"kubernetes.io/projected/404c3b8d-81fd-4aad-9841-3441cbb2d3f8-kube-api-access-xwgxj\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:25:08.847863 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:08.847828    2550 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/404c3b8d-81fd-4aad-9841-3441cbb2d3f8-must-gather-output\") on node \"ip-10-0-131-73.ec2.internal\" DevicePath \"\""
May 28 01:25:09.051299 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:09.051253    2550 status_manager.go:895] "Failed to get status for pod" podUID="404c3b8d-81fd-4aad-9841-3441cbb2d3f8" pod="openshift-must-gather-4q6f4/must-gather-gknx5" err="pods \"must-gather-gknx5\" is forbidden: User \"system:node:ip-10-0-131-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-4q6f4\": no relationship found between node 'ip-10-0-131-73.ec2.internal' and this object"
May 28 01:25:09.325305 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:09.324850    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="404c3b8d-81fd-4aad-9841-3441cbb2d3f8" path="/var/lib/kubelet/pods/404c3b8d-81fd-4aad-9841-3441cbb2d3f8/volumes"
May 28 01:25:09.429977 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:09.429946    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0c7ad1ef-47c3-4758-8177-e329a2a73a61/alertmanager/0.log"
May 28 01:25:09.454893 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:09.454860    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0c7ad1ef-47c3-4758-8177-e329a2a73a61/config-reloader/0.log"
May 28 01:25:09.477092 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:09.477027    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0c7ad1ef-47c3-4758-8177-e329a2a73a61/kube-rbac-proxy-web/0.log"
May 28 01:25:09.503432 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:09.503359    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0c7ad1ef-47c3-4758-8177-e329a2a73a61/kube-rbac-proxy/0.log"
May 28 01:25:09.525077 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:09.525038    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0c7ad1ef-47c3-4758-8177-e329a2a73a61/kube-rbac-proxy-metric/0.log"
May 28 01:25:09.552779 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:09.552750    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0c7ad1ef-47c3-4758-8177-e329a2a73a61/prom-label-proxy/0.log"
May 28 01:25:09.577172 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:09.577085    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0c7ad1ef-47c3-4758-8177-e329a2a73a61/init-config-reloader/0.log"
May 28 01:25:09.708495 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:09.708462    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-75b8cbf7b5-jmflm_afe0dd73-a436-4f93-9420-fdd562172910/metrics-server/0.log"
May 28 01:25:09.762314 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:09.762281    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4w7vg_e1a012c0-f76b-451c-b9fc-cd5755adebdd/node-exporter/0.log"
May 28 01:25:09.782229 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:09.782197    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4w7vg_e1a012c0-f76b-451c-b9fc-cd5755adebdd/kube-rbac-proxy/0.log"
May 28 01:25:09.806946 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:09.806912    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4w7vg_e1a012c0-f76b-451c-b9fc-cd5755adebdd/init-textfile/0.log"
May 28 01:25:10.291887 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:10.291861    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-54554f4f95-prf65_f778803e-1d0e-4983-84ab-33f29e7b141a/telemeter-client/0.log"
May 28 01:25:10.312543 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:10.312513    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-54554f4f95-prf65_f778803e-1d0e-4983-84ab-33f29e7b141a/reload/0.log"
May 28 01:25:10.332675 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:10.332643    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-54554f4f95-prf65_f778803e-1d0e-4983-84ab-33f29e7b141a/kube-rbac-proxy/0.log"
May 28 01:25:12.401091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:12.401024    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5d855856b4-7qkd5_d4a9b823-a3a8-43b5-b620-a436a598a939/console/0.log"
May 28 01:25:13.017113 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.017071    2550 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-m2w8l/perf-node-gather-daemonset-587tv"]
May 28 01:25:13.017512 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.017488    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="404c3b8d-81fd-4aad-9841-3441cbb2d3f8" containerName="gather"
May 28 01:25:13.017612 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.017515    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="404c3b8d-81fd-4aad-9841-3441cbb2d3f8" containerName="gather"
May 28 01:25:13.017612 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.017548    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="404c3b8d-81fd-4aad-9841-3441cbb2d3f8" containerName="copy"
May 28 01:25:13.017612 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.017558    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="404c3b8d-81fd-4aad-9841-3441cbb2d3f8" containerName="copy"
May 28 01:25:13.017775 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.017628    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="404c3b8d-81fd-4aad-9841-3441cbb2d3f8" containerName="copy"
May 28 01:25:13.017775 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.017642    2550 memory_manager.go:356] "RemoveStaleState removing state" podUID="404c3b8d-81fd-4aad-9841-3441cbb2d3f8" containerName="gather"
May 28 01:25:13.022618 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.022588    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m2w8l/perf-node-gather-daemonset-587tv"
May 28 01:25:13.030259 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.030215    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m2w8l/perf-node-gather-daemonset-587tv"]
May 28 01:25:13.186894 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.186854    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/429170c5-ae29-4e68-aaed-3c3085452221-podres\") pod \"perf-node-gather-daemonset-587tv\" (UID: \"429170c5-ae29-4e68-aaed-3c3085452221\") " pod="openshift-must-gather-m2w8l/perf-node-gather-daemonset-587tv"
May 28 01:25:13.187091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.186926    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/429170c5-ae29-4e68-aaed-3c3085452221-lib-modules\") pod \"perf-node-gather-daemonset-587tv\" (UID: \"429170c5-ae29-4e68-aaed-3c3085452221\") " pod="openshift-must-gather-m2w8l/perf-node-gather-daemonset-587tv"
May 28 01:25:13.187091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.187016    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/429170c5-ae29-4e68-aaed-3c3085452221-sys\") pod \"perf-node-gather-daemonset-587tv\" (UID: \"429170c5-ae29-4e68-aaed-3c3085452221\") " pod="openshift-must-gather-m2w8l/perf-node-gather-daemonset-587tv"
May 28 01:25:13.187091 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.187043    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9hsp\" (UniqueName: \"kubernetes.io/projected/429170c5-ae29-4e68-aaed-3c3085452221-kube-api-access-z9hsp\") pod \"perf-node-gather-daemonset-587tv\" (UID: \"429170c5-ae29-4e68-aaed-3c3085452221\") " pod="openshift-must-gather-m2w8l/perf-node-gather-daemonset-587tv"
May 28 01:25:13.187285 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.187166    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/429170c5-ae29-4e68-aaed-3c3085452221-proc\") pod \"perf-node-gather-daemonset-587tv\" (UID: \"429170c5-ae29-4e68-aaed-3c3085452221\") " pod="openshift-must-gather-m2w8l/perf-node-gather-daemonset-587tv"
May 28 01:25:13.288107 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.288016    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/429170c5-ae29-4e68-aaed-3c3085452221-proc\") pod \"perf-node-gather-daemonset-587tv\" (UID: \"429170c5-ae29-4e68-aaed-3c3085452221\") " pod="openshift-must-gather-m2w8l/perf-node-gather-daemonset-587tv"
May 28 01:25:13.288107 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.288065    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/429170c5-ae29-4e68-aaed-3c3085452221-podres\") pod \"perf-node-gather-daemonset-587tv\" (UID: \"429170c5-ae29-4e68-aaed-3c3085452221\") " pod="openshift-must-gather-m2w8l/perf-node-gather-daemonset-587tv"
May 28 01:25:13.288107 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.288085    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/429170c5-ae29-4e68-aaed-3c3085452221-lib-modules\") pod \"perf-node-gather-daemonset-587tv\" (UID: \"429170c5-ae29-4e68-aaed-3c3085452221\") " pod="openshift-must-gather-m2w8l/perf-node-gather-daemonset-587tv"
May 28 01:25:13.288388 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.288122    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/429170c5-ae29-4e68-aaed-3c3085452221-sys\") pod \"perf-node-gather-daemonset-587tv\" (UID: \"429170c5-ae29-4e68-aaed-3c3085452221\") " pod="openshift-must-gather-m2w8l/perf-node-gather-daemonset-587tv"
May 28 01:25:13.288388 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.288138    2550 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z9hsp\" (UniqueName: \"kubernetes.io/projected/429170c5-ae29-4e68-aaed-3c3085452221-kube-api-access-z9hsp\") pod \"perf-node-gather-daemonset-587tv\" (UID: \"429170c5-ae29-4e68-aaed-3c3085452221\") " pod="openshift-must-gather-m2w8l/perf-node-gather-daemonset-587tv"
May 28 01:25:13.288388 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.288155    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/429170c5-ae29-4e68-aaed-3c3085452221-proc\") pod \"perf-node-gather-daemonset-587tv\" (UID: \"429170c5-ae29-4e68-aaed-3c3085452221\") " pod="openshift-must-gather-m2w8l/perf-node-gather-daemonset-587tv"
May 28 01:25:13.288388 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.288185    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/429170c5-ae29-4e68-aaed-3c3085452221-podres\") pod \"perf-node-gather-daemonset-587tv\" (UID: \"429170c5-ae29-4e68-aaed-3c3085452221\") " pod="openshift-must-gather-m2w8l/perf-node-gather-daemonset-587tv"
May 28 01:25:13.288388 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.288227    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/429170c5-ae29-4e68-aaed-3c3085452221-sys\") pod \"perf-node-gather-daemonset-587tv\" (UID: \"429170c5-ae29-4e68-aaed-3c3085452221\") " pod="openshift-must-gather-m2w8l/perf-node-gather-daemonset-587tv"
May 28 01:25:13.288388 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.288309    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/429170c5-ae29-4e68-aaed-3c3085452221-lib-modules\") pod \"perf-node-gather-daemonset-587tv\" (UID: \"429170c5-ae29-4e68-aaed-3c3085452221\") " pod="openshift-must-gather-m2w8l/perf-node-gather-daemonset-587tv"
May 28 01:25:13.295862 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.295838    2550 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9hsp\" (UniqueName: \"kubernetes.io/projected/429170c5-ae29-4e68-aaed-3c3085452221-kube-api-access-z9hsp\") pod \"perf-node-gather-daemonset-587tv\" (UID: \"429170c5-ae29-4e68-aaed-3c3085452221\") " pod="openshift-must-gather-m2w8l/perf-node-gather-daemonset-587tv"
May 28 01:25:13.335244 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.335216    2550 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m2w8l/perf-node-gather-daemonset-587tv"
May 28 01:25:13.482025 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.481992    2550 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m2w8l/perf-node-gather-daemonset-587tv"]
May 28 01:25:13.485079 ip-10-0-131-73 kubenswrapper[2550]: W0528 01:25:13.485053    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod429170c5_ae29_4e68_aaed_3c3085452221.slice/crio-a6bad57eb0a546cc0429408aa372991ef94506d149a9d2c2dea18bb94763259e WatchSource:0}: Error finding container a6bad57eb0a546cc0429408aa372991ef94506d149a9d2c2dea18bb94763259e: Status 404 returned error can't find the container with id a6bad57eb0a546cc0429408aa372991ef94506d149a9d2c2dea18bb94763259e
May 28 01:25:13.504305 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.504283    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-57vdg_703148ea-2824-4a27-ab48-9ace83cd9f1b/dns/0.log"
May 28 01:25:13.524764 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.524738    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-57vdg_703148ea-2824-4a27-ab48-9ace83cd9f1b/kube-rbac-proxy/0.log"
May 28 01:25:13.638106 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.638079    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-m47th_3f79c5d7-ac7c-4ef9-ada8-6d640a7c0ba7/dns-node-resolver/0.log"
May 28 01:25:13.753558 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.753523    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m2w8l/perf-node-gather-daemonset-587tv" event={"ID":"429170c5-ae29-4e68-aaed-3c3085452221","Type":"ContainerStarted","Data":"d234bc1a35c8ae88825243d2f4f1c32faa22ae2f112e6a65def56c5cf3f1ebe5"}
May 28 01:25:13.753558 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.753562    2550 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m2w8l/perf-node-gather-daemonset-587tv" event={"ID":"429170c5-ae29-4e68-aaed-3c3085452221","Type":"ContainerStarted","Data":"a6bad57eb0a546cc0429408aa372991ef94506d149a9d2c2dea18bb94763259e"}
May 28 01:25:13.753787 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.753653    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-m2w8l/perf-node-gather-daemonset-587tv"
May 28 01:25:13.768931 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:13.768889    2550 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-m2w8l/perf-node-gather-daemonset-587tv" podStartSLOduration=0.768876361 podStartE2EDuration="768.876361ms" podCreationTimestamp="2026-05-28 01:25:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 01:25:13.76778482 +0000 UTC m=+1773.048741904" watchObservedRunningTime="2026-05-28 01:25:13.768876361 +0000 UTC m=+1773.049833476"
May 28 01:25:14.066212 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:14.066158    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5997b765f9-fcpb7_4ff8030f-1fcd-4ac0-9e73-cb26a0e48559/registry/0.log"
May 28 01:25:14.107456 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:14.107409    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qxmtt_3a19458a-03be-4a10-a254-cec3229f5266/node-ca/0.log"
May 28 01:25:15.167215 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:15.167177    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gbczp_52d52ce4-3bef-48d9-af17-cd6d77829482/serve-healthcheck-canary/0.log"
May 28 01:25:15.663542 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:15.663459    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tvpvm_f0a88156-df80-4562-b586-8cf16d46c000/kube-rbac-proxy/0.log"
May 28 01:25:15.683114 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:15.683085    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tvpvm_f0a88156-df80-4562-b586-8cf16d46c000/exporter/0.log"
May 28 01:25:15.703137 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:15.703112    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tvpvm_f0a88156-df80-4562-b586-8cf16d46c000/extractor/0.log"
May 28 01:25:19.766641 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:19.766603    2550 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-m2w8l/perf-node-gather-daemonset-587tv"
May 28 01:25:21.033338 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:21.033309    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qn2qw_8ee280af-d845-48ce-9d6e-1fd1baad857d/kube-multus-additional-cni-plugins/0.log"
May 28 01:25:21.054092 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:21.054066    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qn2qw_8ee280af-d845-48ce-9d6e-1fd1baad857d/egress-router-binary-copy/0.log"
May 28 01:25:21.073692 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:21.073653    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qn2qw_8ee280af-d845-48ce-9d6e-1fd1baad857d/cni-plugins/0.log"
May 28 01:25:21.095877 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:21.095849    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qn2qw_8ee280af-d845-48ce-9d6e-1fd1baad857d/bond-cni-plugin/0.log"
May 28 01:25:21.118076 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:21.118049    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qn2qw_8ee280af-d845-48ce-9d6e-1fd1baad857d/routeoverride-cni/0.log"
May 28 01:25:21.138201 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:21.138171    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qn2qw_8ee280af-d845-48ce-9d6e-1fd1baad857d/whereabouts-cni-bincopy/0.log"
May 28 01:25:21.158750 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:21.158717    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qn2qw_8ee280af-d845-48ce-9d6e-1fd1baad857d/whereabouts-cni/0.log"
May 28 01:25:21.222077 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:21.222048    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vxdn6_d829d8f9-26d3-4bab-b318-b59320d5a67e/kube-multus/0.log"
May 28 01:25:21.292007 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:21.291980    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-f8h8b_3d04c1e7-069a-413e-a9f6-d87a8095844d/network-metrics-daemon/0.log"
May 28 01:25:21.311615 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:21.311589    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-f8h8b_3d04c1e7-069a-413e-a9f6-d87a8095844d/kube-rbac-proxy/0.log"
May 28 01:25:22.194707 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:22.194667    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9z98m_6671c1c1-b674-4a1d-9011-1ca22724b234/ovn-controller/0.log"
May 28 01:25:22.212486 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:22.212447    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9z98m_6671c1c1-b674-4a1d-9011-1ca22724b234/ovn-acl-logging/0.log"
May 28 01:25:22.226956 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:22.226920    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9z98m_6671c1c1-b674-4a1d-9011-1ca22724b234/ovn-acl-logging/1.log"
May 28 01:25:22.245455 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:22.245403    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9z98m_6671c1c1-b674-4a1d-9011-1ca22724b234/kube-rbac-proxy-node/0.log"
May 28 01:25:22.269984 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:22.269915    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9z98m_6671c1c1-b674-4a1d-9011-1ca22724b234/kube-rbac-proxy-ovn-metrics/0.log"
May 28 01:25:22.291932 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:22.291901    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9z98m_6671c1c1-b674-4a1d-9011-1ca22724b234/northd/0.log"
May 28 01:25:22.313704 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:22.313681    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9z98m_6671c1c1-b674-4a1d-9011-1ca22724b234/nbdb/0.log"
May 28 01:25:22.338218 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:22.338146    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9z98m_6671c1c1-b674-4a1d-9011-1ca22724b234/sbdb/0.log"
May 28 01:25:22.493193 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:22.493155    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9z98m_6671c1c1-b674-4a1d-9011-1ca22724b234/ovnkube-controller/0.log"
May 28 01:25:24.124450 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:24.124399    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-l9q4k_6fa8fafd-34d7-4feb-8a22-7b1f3fcba8a7/network-check-target-container/0.log"
May 28 01:25:25.017569 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:25.017541    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-nj698_a4ecd41b-62cb-4336-9b2b-d082dd27c847/iptables-alerter/0.log"
May 28 01:25:25.648956 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:25.648925    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-z8xgs_ba57d944-f539-4167-be8f-dd6a88026256/tuned/0.log"
May 28 01:25:28.898659 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:28.898634    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-76lgg_92ed3e71-738c-462d-b98e-44ae3f9a005b/csi-driver/0.log"
May 28 01:25:28.918360 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:28.918336    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-76lgg_92ed3e71-738c-462d-b98e-44ae3f9a005b/csi-node-driver-registrar/0.log"
May 28 01:25:28.938278 ip-10-0-131-73 kubenswrapper[2550]: I0528 01:25:28.938253    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-76lgg_92ed3e71-738c-462d-b98e-44ae3f9a005b/csi-liveness-probe/0.log"