May 28 15:59:11.305643 ip-10-0-133-120 systemd[1]: Starting Kubernetes Kubelet...
May 28 15:59:11.817525 ip-10-0-133-120 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 15:59:11.817525 ip-10-0-133-120 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 28 15:59:11.817525 ip-10-0-133-120 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 15:59:11.817525 ip-10-0-133-120 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 28 15:59:11.817525 ip-10-0-133-120 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 15:59:11.819701 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.819565    2559 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 28 15:59:11.826066 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826039    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 15:59:11.826066 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826062    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 15:59:11.826066 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826066    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 15:59:11.826066 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826070    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 15:59:11.826066 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826073    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 15:59:11.826273 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826077    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 15:59:11.826273 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826080    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 15:59:11.826273 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826083    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 15:59:11.826273 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826086    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 15:59:11.826273 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826089    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 15:59:11.826273 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826091    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 15:59:11.826273 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826094    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 15:59:11.826273 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826097    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 15:59:11.826273 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826100    2559 feature_gate.go:328] unrecognized feature gate: Example
May 28 15:59:11.826273 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826103    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 15:59:11.826273 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826105    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 15:59:11.826273 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826108    2559 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 15:59:11.826273 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826111    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 15:59:11.826273 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826113    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 15:59:11.826273 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826116    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 15:59:11.826273 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826118    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 15:59:11.826273 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826121    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 15:59:11.826273 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826123    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 15:59:11.826273 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826126    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 15:59:11.826273 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826129    2559 feature_gate.go:328] unrecognized feature gate: Example2
May 28 15:59:11.826764 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826131    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 15:59:11.826764 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826134    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 15:59:11.826764 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826144    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 15:59:11.826764 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826147    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 15:59:11.826764 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826150    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 15:59:11.826764 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826152    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 15:59:11.826764 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826155    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 15:59:11.826764 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826158    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 15:59:11.826764 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826160    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 15:59:11.826764 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826165    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 15:59:11.826764 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826169    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 15:59:11.826764 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826172    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 15:59:11.826764 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826174    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 15:59:11.826764 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826179    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 15:59:11.826764 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826182    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 15:59:11.826764 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826187    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 15:59:11.826764 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826191    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 15:59:11.826764 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826194    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 15:59:11.826764 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826197    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 15:59:11.827270 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826200    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 15:59:11.827270 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826203    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 15:59:11.827270 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826206    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 15:59:11.827270 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826208    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 15:59:11.827270 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826211    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 15:59:11.827270 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826214    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 15:59:11.827270 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826216    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 15:59:11.827270 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826219    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 15:59:11.827270 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826222    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 15:59:11.827270 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826224    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 15:59:11.827270 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826227    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 15:59:11.827270 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826229    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 15:59:11.827270 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826232    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 15:59:11.827270 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826234    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 15:59:11.827270 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826237    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 15:59:11.827270 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826240    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 15:59:11.827270 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826242    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 15:59:11.827270 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826245    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 15:59:11.827270 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826247    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 15:59:11.827270 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826250    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 15:59:11.827768 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826253    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 15:59:11.827768 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826256    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 15:59:11.827768 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826258    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 15:59:11.827768 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826262    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 15:59:11.827768 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826264    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 15:59:11.827768 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826267    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 15:59:11.827768 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826271    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 15:59:11.827768 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826274    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 15:59:11.827768 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826277    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 15:59:11.827768 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826280    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 15:59:11.827768 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826282    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 15:59:11.827768 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826285    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 15:59:11.827768 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826288    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 15:59:11.827768 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826290    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 15:59:11.827768 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826294    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 15:59:11.827768 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826297    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 15:59:11.827768 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826299    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 15:59:11.827768 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826302    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 15:59:11.827768 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826305    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 15:59:11.827768 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826307    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 15:59:11.828250 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826310    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 15:59:11.828250 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826312    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 15:59:11.828250 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826730    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 15:59:11.828250 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826735    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 15:59:11.828250 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826739    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 15:59:11.828250 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826742    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 15:59:11.828250 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826744    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 15:59:11.828250 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826747    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 15:59:11.828250 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826750    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 15:59:11.828250 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826753    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 15:59:11.828250 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826756    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 15:59:11.828250 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826758    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 15:59:11.828250 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826761    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 15:59:11.828250 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826764    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 15:59:11.828250 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826766    2559 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 15:59:11.828250 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826769    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 15:59:11.828250 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826772    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 15:59:11.828250 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826775    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 15:59:11.828250 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826778    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 15:59:11.828250 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826781    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 15:59:11.828747 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826783    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 15:59:11.828747 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826786    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 15:59:11.828747 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826789    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 15:59:11.828747 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826792    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 15:59:11.828747 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826795    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 15:59:11.828747 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826797    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 15:59:11.828747 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826801    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 15:59:11.828747 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826803    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 15:59:11.828747 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826806    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 15:59:11.828747 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826808    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 15:59:11.828747 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826811    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 15:59:11.828747 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826813    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 15:59:11.828747 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826815    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 15:59:11.828747 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826818    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 15:59:11.828747 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826820    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 15:59:11.828747 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826823    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 15:59:11.828747 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826825    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 15:59:11.828747 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826828    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 15:59:11.828747 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826830    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 15:59:11.829211 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826833    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 15:59:11.829211 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826835    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 15:59:11.829211 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826837    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 15:59:11.829211 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826840    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 15:59:11.829211 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826843    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 15:59:11.829211 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826845    2559 feature_gate.go:328] unrecognized feature gate: Example
May 28 15:59:11.829211 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826848    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 15:59:11.829211 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826851    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 15:59:11.829211 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826853    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 15:59:11.829211 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826856    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 15:59:11.829211 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826859    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 15:59:11.829211 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826863    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 15:59:11.829211 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826867    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 15:59:11.829211 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826871    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 15:59:11.829211 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826874    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 15:59:11.829211 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826877    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 15:59:11.829211 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826880    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 15:59:11.829211 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826883    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 15:59:11.829211 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826885    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 15:59:11.829211 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826888    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 15:59:11.829719 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826891    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 15:59:11.829719 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826893    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 15:59:11.829719 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826896    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 15:59:11.829719 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826898    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 15:59:11.829719 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826901    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 15:59:11.829719 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826903    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 15:59:11.829719 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826907    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 15:59:11.829719 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826910    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 15:59:11.829719 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826912    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 15:59:11.829719 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826916    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 15:59:11.829719 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826918    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 15:59:11.829719 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826921    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 15:59:11.829719 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826923    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 15:59:11.829719 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826926    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 15:59:11.829719 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826928    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 15:59:11.829719 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826931    2559 feature_gate.go:328] unrecognized feature gate: Example2
May 28 15:59:11.829719 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826934    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 15:59:11.829719 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826937    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 15:59:11.829719 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826940    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 15:59:11.829719 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826942    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 15:59:11.830195 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826945    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 15:59:11.830195 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826948    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 15:59:11.830195 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826951    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 15:59:11.830195 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826953    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 15:59:11.830195 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826956    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 15:59:11.830195 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826958    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 15:59:11.830195 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826961    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 15:59:11.830195 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826963    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 15:59:11.830195 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.826966    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 15:59:11.830195 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827038    2559 flags.go:64] FLAG: --address="0.0.0.0"
May 28 15:59:11.830195 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827045    2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 28 15:59:11.830195 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827053    2559 flags.go:64] FLAG: --anonymous-auth="true"
May 28 15:59:11.830195 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827058    2559 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 28 15:59:11.830195 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827064    2559 flags.go:64] FLAG: --authentication-token-webhook="false"
May 28 15:59:11.830195 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827067    2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 28 15:59:11.830195 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827072    2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 28 15:59:11.830195 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827076    2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 28 15:59:11.830195 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827079    2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 28 15:59:11.830195 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827082    2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 28 15:59:11.830195 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827086    2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 28 15:59:11.830195 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827089    2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 28 15:59:11.830195 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827093    2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 28 15:59:11.830749 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827096    2559 flags.go:64] FLAG: --cgroup-root=""
May 28 15:59:11.830749 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827099    2559 flags.go:64] FLAG: --cgroups-per-qos="true"
May 28 15:59:11.830749 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827102    2559 flags.go:64] FLAG: --client-ca-file=""
May 28 15:59:11.830749 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827105    2559 flags.go:64] FLAG: --cloud-config=""
May 28 15:59:11.830749 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827108    2559 flags.go:64] FLAG: --cloud-provider="external"
May 28 15:59:11.830749 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827111    2559 flags.go:64] FLAG: --cluster-dns="[]"
May 28 15:59:11.830749 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827115    2559 flags.go:64] FLAG: --cluster-domain=""
May 28 15:59:11.830749 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827118    2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 28 15:59:11.830749 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827122    2559 flags.go:64] FLAG: --config-dir=""
May 28 15:59:11.830749 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827125    2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 28 15:59:11.830749 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827129    2559 flags.go:64] FLAG: --container-log-max-files="5"
May 28 15:59:11.830749 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827133    2559 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 28 15:59:11.830749 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827136    2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 28 15:59:11.830749 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827139    2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 28 15:59:11.830749 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827142    2559 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 28 15:59:11.830749 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827145    2559 flags.go:64] FLAG: --contention-profiling="false"
May 28 15:59:11.830749 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827149    2559 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 28 15:59:11.830749 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827152    2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 28 15:59:11.830749 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827155    2559 flags.go:64] FLAG: --cpu-manager-policy="none"
May 28 15:59:11.830749 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827158    2559 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 28 15:59:11.830749 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827162    2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 28 15:59:11.830749 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827165    2559 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 28 15:59:11.830749 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827168    2559 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 28 15:59:11.830749 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827171    2559 flags.go:64] FLAG: --enable-load-reader="false"
May 28 15:59:11.830749 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827175    2559 flags.go:64] FLAG: --enable-server="true"
May 28 15:59:11.831386 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827178    2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 28 15:59:11.831386 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827183    2559 flags.go:64] FLAG: --event-burst="100"
May 28 15:59:11.831386 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827186    2559 flags.go:64] FLAG: --event-qps="50"
May 28 15:59:11.831386 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827189    2559 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 28 15:59:11.831386 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827193    2559 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 28 15:59:11.831386 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827196    2559 flags.go:64] FLAG: --eviction-hard=""
May 28 15:59:11.831386 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827200    2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 28 15:59:11.831386 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827203    2559 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 28 15:59:11.831386 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827206    2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 28 15:59:11.831386 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827210    2559 flags.go:64] FLAG: --eviction-soft=""
May 28 15:59:11.831386 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827213    2559 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 28 15:59:11.831386 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827216    2559 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 28 15:59:11.831386 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827219    2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 28 15:59:11.831386 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827222    2559 flags.go:64] FLAG: --experimental-mounter-path=""
May 28 15:59:11.831386 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827225    2559 flags.go:64] FLAG: --fail-cgroupv1="false"
May 28 15:59:11.831386 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827228    2559 flags.go:64] FLAG: --fail-swap-on="true"
May 28 15:59:11.831386 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827230    2559 flags.go:64] FLAG: --feature-gates=""
May 28 15:59:11.831386 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827235    2559 flags.go:64] FLAG: --file-check-frequency="20s"
May 28 15:59:11.831386 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827238    2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 28 15:59:11.831386 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827241    2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 28 15:59:11.831386 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827244    2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 28 15:59:11.831386 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827247    2559 flags.go:64] FLAG: --healthz-port="10248"
May 28 15:59:11.831386 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827250    2559 flags.go:64] FLAG: --help="false"
May 28 15:59:11.831386 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827253    2559 flags.go:64] FLAG: --hostname-override="ip-10-0-133-120.ec2.internal"
May 28 15:59:11.831386 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827256    2559 flags.go:64] FLAG: --housekeeping-interval="10s"
May 28 15:59:11.832057 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827259    2559 flags.go:64] FLAG: --http-check-frequency="20s"
May 28 15:59:11.832057 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827262    2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 28 15:59:11.832057 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827266    2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 28 15:59:11.832057 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827269    2559 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 28 15:59:11.832057 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827272    2559 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 28 15:59:11.832057 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827275    2559 flags.go:64] FLAG: --image-service-endpoint=""
May 28 15:59:11.832057 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827278    2559 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 28 15:59:11.832057 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827281    2559 flags.go:64] FLAG: --kube-api-burst="100"
May 28 15:59:11.832057 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827284    2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 28 15:59:11.832057 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827288    2559 flags.go:64] FLAG: --kube-api-qps="50"
May 28 15:59:11.832057 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827290    2559 flags.go:64] FLAG: --kube-reserved=""
May 28 15:59:11.832057 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827293    2559 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 28 15:59:11.832057 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827296    2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 28 15:59:11.832057 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827299    2559 flags.go:64] FLAG: --kubelet-cgroups=""
May 28 15:59:11.832057 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827302    2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 28 15:59:11.832057 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827305    2559 flags.go:64] FLAG: --lock-file=""
May 28 15:59:11.832057 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827308    2559 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 28 15:59:11.832057 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827311    2559 flags.go:64] FLAG: --log-flush-frequency="5s"
May 28 15:59:11.832057 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827314    2559 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 28 15:59:11.832057 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827319    2559 flags.go:64] FLAG: --log-json-split-stream="false"
May 28 15:59:11.832057 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827322    2559 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 28 15:59:11.832057 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827339    2559 flags.go:64] FLAG: --log-text-split-stream="false"
May 28 15:59:11.832057 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827341    2559 flags.go:64] FLAG: --logging-format="text"
May 28 15:59:11.832694 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827345    2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 28 15:59:11.832694 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827349    2559 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 28 15:59:11.832694 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827352    2559 flags.go:64] FLAG: --manifest-url=""
May 28 15:59:11.832694 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827355    2559 flags.go:64] FLAG: --manifest-url-header=""
May 28 15:59:11.832694 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827359    2559 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 28 15:59:11.832694 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827363    2559 flags.go:64] FLAG: --max-open-files="1000000"
May 28 15:59:11.832694 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827367    2559 flags.go:64] FLAG: --max-pods="110"
May 28 15:59:11.832694 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827370    2559 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 28 15:59:11.832694 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827373    2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 28 15:59:11.832694 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827376    2559 flags.go:64] FLAG: --memory-manager-policy="None"
May 28 15:59:11.832694 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827379    2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 28 15:59:11.832694 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827382    2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 28 15:59:11.832694 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827385    2559 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 28 15:59:11.832694 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827388    2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 28 15:59:11.832694 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827396    2559 flags.go:64] FLAG: --node-status-max-images="50"
May 28 15:59:11.832694 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827399    2559 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 28 15:59:11.832694 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827402    2559 flags.go:64] FLAG: --oom-score-adj="-999"
May 28 15:59:11.832694 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827405    2559 flags.go:64] FLAG: --pod-cidr=""
May 28 15:59:11.832694 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827408    2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc"
May 28 15:59:11.832694 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827413    2559 flags.go:64] FLAG: --pod-manifest-path=""
May 28 15:59:11.832694 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827417    2559 flags.go:64] FLAG: --pod-max-pids="-1"
May 28 15:59:11.832694 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827420    2559 flags.go:64] FLAG: --pods-per-core="0"
May 28 15:59:11.832694 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827422    2559 flags.go:64] FLAG: --port="10250"
May 28 15:59:11.832694 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827425    2559 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 28 15:59:11.833273 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827432    2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09d9332529a1b3138"
May 28 15:59:11.833273 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827435    2559 flags.go:64] FLAG: --qos-reserved=""
May 28 15:59:11.833273 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827438    2559 flags.go:64] FLAG: --read-only-port="10255"
May 28 15:59:11.833273 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827441    2559 flags.go:64] FLAG: --register-node="true"
May 28 15:59:11.833273 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827444    2559 flags.go:64] FLAG: --register-schedulable="true"
May 28 15:59:11.833273 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827447    2559 flags.go:64] FLAG: --register-with-taints=""
May 28 15:59:11.833273 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827451    2559 flags.go:64] FLAG: --registry-burst="10"
May 28 15:59:11.833273 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827454    2559 flags.go:64] FLAG: --registry-qps="5"
May 28 15:59:11.833273 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827457    2559 flags.go:64] FLAG: --reserved-cpus=""
May 28 15:59:11.833273 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827460    2559 flags.go:64] FLAG: --reserved-memory=""
May 28 15:59:11.833273 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827463    2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 28 15:59:11.833273 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827466    2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 28 15:59:11.833273 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827469    2559 flags.go:64] FLAG: --rotate-certificates="false"
May 28 15:59:11.833273 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827472    2559 flags.go:64] FLAG: --rotate-server-certificates="false"
May 28 15:59:11.833273 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827475    2559 flags.go:64] FLAG: --runonce="false"
May 28 15:59:11.833273 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827478    2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 28 15:59:11.833273 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827481    2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 28 15:59:11.833273 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827484    2559 flags.go:64] FLAG: --seccomp-default="false"
May 28 15:59:11.833273 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827487    2559 flags.go:64] FLAG: --serialize-image-pulls="true"
May 28 15:59:11.833273 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827489    2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 28 15:59:11.833273 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827492    2559 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 28 15:59:11.833273 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827495    2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 28 15:59:11.833273 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827498    2559 flags.go:64] FLAG: --storage-driver-password="root"
May 28 15:59:11.833273 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827501    2559 flags.go:64] FLAG: --storage-driver-secure="false"
May 28 15:59:11.833273 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827504    2559 flags.go:64] FLAG: --storage-driver-table="stats"
May 28 15:59:11.833273 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827507    2559 flags.go:64] FLAG: --storage-driver-user="root"
May 28 15:59:11.833951 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827510    2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 28 15:59:11.833951 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827513    2559 flags.go:64] FLAG: --sync-frequency="1m0s"
May 28 15:59:11.833951 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827516    2559 flags.go:64] FLAG: --system-cgroups=""
May 28 15:59:11.833951 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827519    2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 28 15:59:11.833951 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827525    2559 flags.go:64] FLAG: --system-reserved-cgroup=""
May 28 15:59:11.833951 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827527    2559 flags.go:64] FLAG: --tls-cert-file=""
May 28 15:59:11.833951 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827531    2559 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 28 15:59:11.833951 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827535    2559 flags.go:64] FLAG: --tls-min-version=""
May 28 15:59:11.833951 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827538    2559 flags.go:64] FLAG: --tls-private-key-file=""
May 28 15:59:11.833951 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827541    2559 flags.go:64] FLAG: --topology-manager-policy="none"
May 28 15:59:11.833951 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827544    2559 flags.go:64] FLAG: --topology-manager-policy-options=""
May 28 15:59:11.833951 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827547    2559 flags.go:64] FLAG: --topology-manager-scope="container"
May 28 15:59:11.833951 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827550    2559 flags.go:64] FLAG: --v="2"
May 28 15:59:11.833951 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827554    2559 flags.go:64] FLAG: --version="false"
May 28 15:59:11.833951 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827558    2559 flags.go:64] FLAG: --vmodule=""
May 28 15:59:11.833951 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827563    2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 28 15:59:11.833951 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.827566    2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 28 15:59:11.833951 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827654    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 15:59:11.833951 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827658    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 15:59:11.833951 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827661    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 15:59:11.833951 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827664    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 15:59:11.833951 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827667    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 15:59:11.833951 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827670    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 15:59:11.833951 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827673    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 15:59:11.834558 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827676    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 15:59:11.834558 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827679    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 15:59:11.834558 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827682    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 15:59:11.834558 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827685    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 15:59:11.834558 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827688    2559 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 15:59:11.834558 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827691    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 15:59:11.834558 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827693    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 15:59:11.834558 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827696    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 15:59:11.834558 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827698    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 15:59:11.834558 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827701    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 15:59:11.834558 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827704    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 15:59:11.834558 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827707    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 15:59:11.834558 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827709    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 15:59:11.834558 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827712    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 15:59:11.834558 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827716    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 15:59:11.834558 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827719    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 15:59:11.834558 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827721    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 15:59:11.834558 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827723    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 15:59:11.834558 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827726    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 15:59:11.835027 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827729    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 15:59:11.835027 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827731    2559 feature_gate.go:328] unrecognized feature gate: Example
May 28 15:59:11.835027 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827733    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 15:59:11.835027 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827736    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 15:59:11.835027 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827738    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 15:59:11.835027 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827744    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 15:59:11.835027 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827746    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 15:59:11.835027 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827750    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 15:59:11.835027 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827752    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 15:59:11.835027 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827755    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 15:59:11.835027 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827757    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 15:59:11.835027 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827760    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 15:59:11.835027 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827763    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 15:59:11.835027 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827765    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 15:59:11.835027 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827768    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 15:59:11.835027 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827770    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 15:59:11.835027 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827773    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 15:59:11.835027 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827776    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 15:59:11.835027 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827779    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 15:59:11.835027 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827781    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 15:59:11.835542 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827784    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 15:59:11.835542 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827787    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 15:59:11.835542 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827789    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 15:59:11.835542 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827792    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 15:59:11.835542 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827795    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 15:59:11.835542 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827797    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 15:59:11.835542 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827801    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 15:59:11.835542 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827805    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 15:59:11.835542 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827808    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 15:59:11.835542 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827811    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 15:59:11.835542 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827814    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 15:59:11.835542 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827817    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 15:59:11.835542 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827819    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 15:59:11.835542 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827822    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 15:59:11.835542 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827824    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 15:59:11.835542 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827827    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 15:59:11.835542 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827829    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 15:59:11.835542 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827833    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 15:59:11.835542 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827836    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 15:59:11.836093 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827838    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 15:59:11.836093 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827841    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 15:59:11.836093 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827843    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 15:59:11.836093 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827846    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 15:59:11.836093 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827848    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 15:59:11.836093 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827852    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 15:59:11.836093 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827856    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 15:59:11.836093 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827858    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 15:59:11.836093 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827861    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 15:59:11.836093 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827863    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 15:59:11.836093 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827866    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 15:59:11.836093 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827869    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 15:59:11.836093 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827871    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 15:59:11.836093 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827874    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 15:59:11.836093 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827877    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 15:59:11.836093 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827879    2559 feature_gate.go:328] unrecognized feature gate: Example2
May 28 15:59:11.836093 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827882    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 15:59:11.836093 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827885    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 15:59:11.836093 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827887    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 15:59:11.836093 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827890    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 15:59:11.836966 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.827893    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 15:59:11.836966 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.828664    2559 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 15:59:11.837719 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.837693    2559 server.go:530] "Kubelet version" kubeletVersion="v1.33.11"
May 28 15:59:11.837719 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.837721    2559 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 28 15:59:11.837861 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837790    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 15:59:11.837861 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837798    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 15:59:11.837861 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837803    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 15:59:11.837861 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837808    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 15:59:11.837861 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837814    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 15:59:11.837861 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837819    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 15:59:11.837861 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837825    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 15:59:11.837861 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837829    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 15:59:11.837861 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837834    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 15:59:11.837861 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837838    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 15:59:11.837861 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837843    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 15:59:11.837861 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837847    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 15:59:11.837861 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837852    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 15:59:11.837861 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837859    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 15:59:11.837861 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837866    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 15:59:11.838533 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837871    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 15:59:11.838533 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837876    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 15:59:11.838533 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837880    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 15:59:11.838533 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837885    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 15:59:11.838533 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837889    2559 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 15:59:11.838533 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837893    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 15:59:11.838533 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837898    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 15:59:11.838533 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837902    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 15:59:11.838533 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837909    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 15:59:11.838533 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837915    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 15:59:11.838533 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837921    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 15:59:11.838533 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837926    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 15:59:11.838533 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837930    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 15:59:11.838533 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837935    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 15:59:11.838533 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837940    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 15:59:11.838533 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837944    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 15:59:11.838533 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837949    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 15:59:11.838533 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837953    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 15:59:11.838533 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837958    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 15:59:11.838533 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837963    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 15:59:11.839302 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837967    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 15:59:11.839302 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837972    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 15:59:11.839302 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837976    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 15:59:11.839302 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837981    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 15:59:11.839302 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837986    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 15:59:11.839302 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837990    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 15:59:11.839302 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837994    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 15:59:11.839302 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.837999    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 15:59:11.839302 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838003    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 15:59:11.839302 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838007    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 15:59:11.839302 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838011    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 15:59:11.839302 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838015    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 15:59:11.839302 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838020    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 15:59:11.839302 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838025    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 15:59:11.839302 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838029    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 15:59:11.839302 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838034    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 15:59:11.839302 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838039    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 15:59:11.839302 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838043    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 15:59:11.839302 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838047    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 15:59:11.839909 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838052    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 15:59:11.839909 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838056    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 15:59:11.839909 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838060    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 15:59:11.839909 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838064    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 15:59:11.839909 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838073    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 15:59:11.839909 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838078    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 15:59:11.839909 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838082    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 15:59:11.839909 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838086    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 15:59:11.839909 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838091    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 15:59:11.839909 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838094    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 15:59:11.839909 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838099    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 15:59:11.839909 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838103    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 15:59:11.839909 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838107    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 15:59:11.839909 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838111    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 15:59:11.839909 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838116    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 15:59:11.839909 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838120    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 15:59:11.839909 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838124    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 15:59:11.839909 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838128    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 15:59:11.839909 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838133    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 15:59:11.839909 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838137    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 15:59:11.840960 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838141    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 15:59:11.840960 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838146    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 15:59:11.840960 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838150    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 15:59:11.840960 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838154    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 15:59:11.840960 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838158    2559 feature_gate.go:328] unrecognized feature gate: Example
May 28 15:59:11.840960 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838162    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 15:59:11.840960 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838167    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 15:59:11.840960 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838171    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 15:59:11.840960 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838175    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 15:59:11.840960 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838179    2559 feature_gate.go:328] unrecognized feature gate: Example2
May 28 15:59:11.840960 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838184    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 15:59:11.840960 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838188    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 15:59:11.840960 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.838197    2559 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 15:59:11.840960 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838381    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 15:59:11.840960 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838390    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 15:59:11.841512 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838396    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 15:59:11.841512 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838401    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 15:59:11.841512 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838406    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 15:59:11.841512 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838411    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 15:59:11.841512 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838416    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 15:59:11.841512 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838421    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 15:59:11.841512 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838427    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 15:59:11.841512 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838431    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 15:59:11.841512 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838436    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 15:59:11.841512 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838440    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 15:59:11.841512 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838444    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 15:59:11.841512 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838449    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 15:59:11.841512 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838454    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 15:59:11.841512 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838459    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 15:59:11.841512 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838464    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 15:59:11.841512 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838468    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 15:59:11.841512 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838472    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 15:59:11.841512 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838477    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 15:59:11.841512 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838481    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 15:59:11.841512 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838485    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 15:59:11.842079 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838489    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 15:59:11.842079 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838493    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 15:59:11.842079 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838498    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 15:59:11.842079 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838502    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 15:59:11.842079 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838507    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 15:59:11.842079 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838511    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 15:59:11.842079 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838515    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 15:59:11.842079 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838520    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 15:59:11.842079 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838524    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 15:59:11.842079 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838528    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 15:59:11.842079 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838533    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 15:59:11.842079 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838537    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 15:59:11.842079 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838541    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 15:59:11.842079 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838545    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 15:59:11.842079 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838549    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 15:59:11.842079 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838554    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 15:59:11.842079 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838558    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 15:59:11.842079 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838562    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 15:59:11.842079 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838566    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 15:59:11.842636 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838570    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 15:59:11.842636 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838575    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 15:59:11.842636 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838579    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 15:59:11.842636 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838583    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 15:59:11.842636 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838588    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 15:59:11.842636 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838592    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 15:59:11.842636 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838596    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 15:59:11.842636 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838600    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 15:59:11.842636 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838606    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 15:59:11.842636 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838614    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 15:59:11.842636 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838618    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 15:59:11.842636 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838648    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 15:59:11.842636 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838655    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 15:59:11.842636 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838660    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 15:59:11.842636 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838665    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 15:59:11.842636 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838669    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 15:59:11.842636 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838674    2559 feature_gate.go:328] unrecognized feature gate: Example2
May 28 15:59:11.842636 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838679    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 15:59:11.842636 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838683    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 15:59:11.842636 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838688    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 15:59:11.843179 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838692    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 15:59:11.843179 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838696    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 15:59:11.843179 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838700    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 15:59:11.843179 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838704    2559 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 15:59:11.843179 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838711    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 15:59:11.843179 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838717    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 15:59:11.843179 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838722    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 15:59:11.843179 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838726    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 15:59:11.843179 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838731    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 15:59:11.843179 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838736    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 15:59:11.843179 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838741    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 15:59:11.843179 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838745    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 15:59:11.843179 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838749    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 15:59:11.843179 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838753    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 15:59:11.843179 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838757    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 15:59:11.843179 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838761    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 15:59:11.843179 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838765    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 15:59:11.843179 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838769    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 15:59:11.843179 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838773    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 15:59:11.843693 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838778    2559 feature_gate.go:328] unrecognized feature gate: Example
May 28 15:59:11.843693 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838782    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 15:59:11.843693 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838786    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 15:59:11.843693 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838791    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 15:59:11.843693 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838795    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 15:59:11.843693 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:11.838799    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 15:59:11.843693 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.838807    2559 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 15:59:11.843693 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.838988    2559 server.go:962] "Client rotation is on, will bootstrap in background"
May 28 15:59:11.843693 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.841949    2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 28 15:59:11.843693 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.843017    2559 server.go:1019] "Starting client certificate rotation"
May 28 15:59:11.843693 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.843120    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 15:59:11.843693 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.843151    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 15:59:11.873415 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.873392    2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 15:59:11.878617 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.878587    2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 15:59:11.894916 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.894893    2559 log.go:25] "Validated CRI v1 runtime API"
May 28 15:59:11.900935 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.900914    2559 log.go:25] "Validated CRI v1 image API"
May 28 15:59:11.901795 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.901777    2559 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 15:59:11.904107 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.904091    2559 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 28 15:59:11.908801 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.908778    2559 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a56462eb-2e56-46d8-b55d-042f57922602:/dev/nvme0n1p3 a739f7dc-80af-407e-84d3-7f677688d967:/dev/nvme0n1p4]
May 28 15:59:11.908871 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.908800    2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 28 15:59:11.913237 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.913127    2559 manager.go:217] Machine: {Timestamp:2026-05-28 15:59:11.912545126 +0000 UTC m=+0.472188615 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3046109 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2fae7aa2c893059564231b7e36fbaa SystemUUID:ec2fae7a-a2c8-9305-9564-231b7e36fbaa BootID:b9335697-e35d-489e-9769-ce70b057cb9e Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:30:7a:8e:38:75 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:30:7a:8e:38:75 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:52:a2:c5:9c:62:58 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 28 15:59:11.913888 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.913878    2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 28 15:59:11.914026 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.914013    2559 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 28 15:59:11.917716 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.917690    2559 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 28 15:59:11.917858 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.917719    2559 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-120.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 28 15:59:11.917908 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.917866    2559 topology_manager.go:138] "Creating topology manager with none policy"
May 28 15:59:11.917908 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.917875    2559 container_manager_linux.go:306] "Creating device plugin manager"
May 28 15:59:11.917908 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.917882    2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 15:59:11.918875 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.918863    2559 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 15:59:11.921089 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.921074    2559 state_mem.go:36] "Initialized new in-memory state store"
May 28 15:59:11.921406 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.921395    2559 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 28 15:59:11.923898 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.923886    2559 kubelet.go:491] "Attempting to sync node with API server"
May 28 15:59:11.923945 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.923908    2559 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 28 15:59:11.923945 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.923921    2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 28 15:59:11.923945 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.923931    2559 kubelet.go:397] "Adding apiserver pod source"
May 28 15:59:11.923945 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.923941    2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 28 15:59:11.925118 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.925104    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 15:59:11.925166 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.925125    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 15:59:11.928610 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.928582    2559 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 28 15:59:11.930911 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.930897    2559 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 28 15:59:11.931861 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.931845    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 28 15:59:11.931907 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.931866    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 28 15:59:11.931907 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.931873    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 28 15:59:11.931907 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.931879    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 28 15:59:11.931907 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.931887    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 28 15:59:11.931907 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.931894    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 28 15:59:11.931907 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.931903    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 28 15:59:11.932067 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.931921    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 28 15:59:11.932067 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.931930    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 28 15:59:11.932067 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.931935    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 28 15:59:11.932067 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.931951    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 28 15:59:11.932412 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.932403    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 28 15:59:11.933470 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.933460    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 28 15:59:11.933514 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.933473    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 28 15:59:11.936477 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.936455    2559 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-120.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 28 15:59:11.937040 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.937028    2559 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 28 15:59:11.937093 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.937063    2559 server.go:1295] "Started kubelet"
May 28 15:59:11.937135 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.937106    2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 28 15:59:11.937204 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.937152    2559 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 28 15:59:11.937267 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.937234    2559 server_v1.go:47] "podresources" method="list" useActivePods=true
May 28 15:59:11.937353 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:11.937308    2559 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-120.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 28 15:59:11.937403 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:11.937308    2559 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 28 15:59:11.937990 ip-10-0-133-120 systemd[1]: Started Kubernetes Kubelet.
May 28 15:59:11.938186 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.938173    2559 server.go:317] "Adding debug handlers to kubelet server"
May 28 15:59:11.938942 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.938925    2559 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 28 15:59:11.941097 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.941081    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zdl8s"
May 28 15:59:11.945541 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:11.944364    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-120.ec2.internal.18b3c5f9391a7015  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-120.ec2.internal,UID:ip-10-0-133-120.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-120.ec2.internal,},FirstTimestamp:2026-05-28 15:59:11.937040405 +0000 UTC m=+0.496683894,LastTimestamp:2026-05-28 15:59:11.937040405 +0000 UTC m=+0.496683894,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-120.ec2.internal,}"
May 28 15:59:11.946738 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.946722    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 28 15:59:11.947460 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.947445    2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 28 15:59:11.947557 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.947546    2559 volume_manager.go:295] "The desired_state_of_world populator starts"
May 28 15:59:11.947603 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.947558    2559 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 28 15:59:11.947726 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.947710    2559 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 28 15:59:11.947789 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.947755    2559 reconstruct.go:97] "Volume reconstruction finished"
May 28 15:59:11.947789 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.947760    2559 reconciler.go:26] "Reconciler: start to sync state"
May 28 15:59:11.950479 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:11.950453    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
May 28 15:59:11.951154 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.951131    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zdl8s"
May 28 15:59:11.951483 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:11.951409    2559 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 28 15:59:11.951552 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.951503    2559 factory.go:153] Registering CRI-O factory
May 28 15:59:11.951552 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.951527    2559 factory.go:223] Registration of the crio container factory successfully
May 28 15:59:11.951656 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.951584    2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 28 15:59:11.951656 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.951592    2559 factory.go:55] Registering systemd factory
May 28 15:59:11.951738 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.951659    2559 factory.go:223] Registration of the systemd container factory successfully
May 28 15:59:11.951738 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.951676    2559 factory.go:103] Registering Raw factory
May 28 15:59:11.951738 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.951694    2559 manager.go:1196] Started watching for new ooms in manager
May 28 15:59:11.954010 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.953993    2559 manager.go:319] Starting recovery of all containers
May 28 15:59:11.954124 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:11.954102    2559 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
May 28 15:59:11.954124 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:11.954102    2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-120.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
May 28 15:59:11.963243 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.963227    2559 manager.go:324] Recovery completed
May 28 15:59:11.967161 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.967148    2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 15:59:11.968998 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.968981    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientMemory"
May 28 15:59:11.969074 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.969010    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasNoDiskPressure"
May 28 15:59:11.969074 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.969028    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientPID"
May 28 15:59:11.969592 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.969578    2559 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 28 15:59:11.969647 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.969592    2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 28 15:59:11.969647 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.969611    2559 state_mem.go:36] "Initialized new in-memory state store"
May 28 15:59:11.973481 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.973469    2559 policy_none.go:49] "None policy: Start"
May 28 15:59:11.973539 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.973485    2559 memory_manager.go:186] "Starting memorymanager" policy="None"
May 28 15:59:11.973539 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:11.973495    2559 state_mem.go:35] "Initializing new in-memory state store"
May 28 15:59:12.024536 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.012973    2559 manager.go:341] "Starting Device Plugin manager"
May 28 15:59:12.024536 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:12.013086    2559 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 28 15:59:12.024536 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.013102    2559 server.go:85] "Starting device plugin registration server"
May 28 15:59:12.024536 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.013367    2559 eviction_manager.go:189] "Eviction manager: starting control loop"
May 28 15:59:12.024536 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.013377    2559 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 28 15:59:12.024536 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.013486    2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 28 15:59:12.024536 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.013566    2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 28 15:59:12.024536 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.013575    2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 28 15:59:12.024536 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:12.014102    2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 28 15:59:12.024536 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:12.014134    2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-120.ec2.internal\" not found"
May 28 15:59:12.068018 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.067933    2559 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 28 15:59:12.069100 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.069083    2559 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 28 15:59:12.069161 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.069108    2559 status_manager.go:230] "Starting to sync pod status with apiserver"
May 28 15:59:12.069161 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.069130    2559 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 28 15:59:12.069161 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.069137    2559 kubelet.go:2451] "Starting kubelet main sync loop"
May 28 15:59:12.069257 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:12.069178    2559 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 28 15:59:12.071700 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.071680    2559 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 15:59:12.113648 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.113601    2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 15:59:12.114726 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.114709    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientMemory"
May 28 15:59:12.114820 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.114743    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasNoDiskPressure"
May 28 15:59:12.114820 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.114757    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientPID"
May 28 15:59:12.114820 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.114779    2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-120.ec2.internal"
May 28 15:59:12.121007 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.120991    2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-120.ec2.internal"
May 28 15:59:12.121047 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:12.121014    2559 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-120.ec2.internal\": node \"ip-10-0-133-120.ec2.internal\" not found"
May 28 15:59:12.136521 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:12.136501    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
May 28 15:59:12.170232 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.170207    2559 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal"]
May 28 15:59:12.170358 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.170270    2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 15:59:12.171102 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.171090    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientMemory"
May 28 15:59:12.171156 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.171114    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasNoDiskPressure"
May 28 15:59:12.171156 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.171125    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientPID"
May 28 15:59:12.172444 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.172431    2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 15:59:12.172589 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.172575    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
May 28 15:59:12.172633 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.172603    2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 15:59:12.173139 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.173124    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientMemory"
May 28 15:59:12.173139 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.173133    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientMemory"
May 28 15:59:12.173213 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.173153    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasNoDiskPressure"
May 28 15:59:12.173213 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.173153    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasNoDiskPressure"
May 28 15:59:12.173213 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.173164    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientPID"
May 28 15:59:12.173213 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.173171    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientPID"
May 28 15:59:12.174439 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.174426    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal"
May 28 15:59:12.174487 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.174450    2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 15:59:12.175103 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.175085    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientMemory"
May 28 15:59:12.175199 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.175112    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasNoDiskPressure"
May 28 15:59:12.175199 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.175127    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientPID"
May 28 15:59:12.207182 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:12.207149    2559 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-120.ec2.internal\" not found" node="ip-10-0-133-120.ec2.internal"
May 28 15:59:12.211615 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:12.211597    2559 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-120.ec2.internal\" not found" node="ip-10-0-133-120.ec2.internal"
May 28 15:59:12.237299 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:12.237271    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
May 28 15:59:12.249258 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.249231    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6e035ff9b129e563067f987de3c32004-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal\" (UID: \"6e035ff9b129e563067f987de3c32004\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
May 28 15:59:12.249372 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.249261    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e035ff9b129e563067f987de3c32004-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal\" (UID: \"6e035ff9b129e563067f987de3c32004\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
May 28 15:59:12.249372 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.249279    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6db7b9c4232c4daf9a5a539bc775452b-config\") pod \"kube-apiserver-proxy-ip-10-0-133-120.ec2.internal\" (UID: \"6db7b9c4232c4daf9a5a539bc775452b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal"
May 28 15:59:12.338194 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:12.338100    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
May 28 15:59:12.349444 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.349421    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6e035ff9b129e563067f987de3c32004-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal\" (UID: \"6e035ff9b129e563067f987de3c32004\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
May 28 15:59:12.349531 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.349448    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e035ff9b129e563067f987de3c32004-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal\" (UID: \"6e035ff9b129e563067f987de3c32004\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
May 28 15:59:12.349531 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.349466    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6db7b9c4232c4daf9a5a539bc775452b-config\") pod \"kube-apiserver-proxy-ip-10-0-133-120.ec2.internal\" (UID: \"6db7b9c4232c4daf9a5a539bc775452b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal"
May 28 15:59:12.349531 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.349523    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e035ff9b129e563067f987de3c32004-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal\" (UID: \"6e035ff9b129e563067f987de3c32004\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
May 28 15:59:12.349634 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.349551    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6e035ff9b129e563067f987de3c32004-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal\" (UID: \"6e035ff9b129e563067f987de3c32004\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
May 28 15:59:12.349634 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.349523    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6db7b9c4232c4daf9a5a539bc775452b-config\") pod \"kube-apiserver-proxy-ip-10-0-133-120.ec2.internal\" (UID: \"6db7b9c4232c4daf9a5a539bc775452b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal"
May 28 15:59:12.438893 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:12.438852    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
May 28 15:59:12.510363 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.510319    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
May 28 15:59:12.515061 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.515042    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal"
May 28 15:59:12.539917 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:12.539871    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
May 28 15:59:12.641040 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:12.640942    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
May 28 15:59:12.741486 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:12.741452    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
May 28 15:59:12.842028 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:12.841980    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
May 28 15:59:12.843086 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.843060    2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 28 15:59:12.843244 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.843229    2559 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 28 15:59:12.889531 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.889506    2559 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 15:59:12.923127 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.923101    2559 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 15:59:12.924362 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.924349    2559 apiserver.go:52] "Watching apiserver"
May 28 15:59:12.934536 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.934505    2559 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 28 15:59:12.934850 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.934823    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-56thk","openshift-dns/node-resolver-ztbwc","openshift-multus/network-metrics-daemon-69vjf","openshift-network-diagnostics/network-check-target-lw6n8","openshift-network-operator/iptables-alerter-8krdd","kube-system/konnectivity-agent-vj78z","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8","openshift-image-registry/node-ca-5dd8s","openshift-multus/multus-additional-cni-plugins-fgmnc","openshift-multus/multus-m8b2m","openshift-ovn-kubernetes/ovnkube-node-bgmxp"]
May 28 15:59:12.937395 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.937373    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:12.939197 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.939182    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ztbwc"
May 28 15:59:12.939703 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.939685    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 28 15:59:12.939808 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.939703    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nlv45\""
May 28 15:59:12.940186 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.940173    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 28 15:59:12.941128 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.941114    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:12.941205 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:12.941184    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69vjf" podUID="bea98d0f-e54c-4da9-a207-f692c8781bea"
May 28 15:59:12.941506 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.941493    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 28 15:59:12.941691 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.941674    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lqwfl\""
May 28 15:59:12.941772 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.941760    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 28 15:59:12.943075 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.943059    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:12.943151 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:12.943114    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lw6n8" podUID="c73529ee-0f31-4285-85c2-51cdfd913e80"
May 28 15:59:12.944991 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.944975    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8krdd"
May 28 15:59:12.946824 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.946810    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 28 15:59:12.946898 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.946852    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 28 15:59:12.946954 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.946917    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vj78z"
May 28 15:59:12.947229 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.947215    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 28 15:59:12.947318 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.947299    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 28 15:59:12.947378 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.947305    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-l8q9v\""
May 28 15:59:12.948175 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.948161    2559 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
May 28 15:59:12.948809 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.948796    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 28 15:59:12.949988 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.949973    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gnxnx\""
May 28 15:59:12.950153 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.950141    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 28 15:59:12.951212 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.951184    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8"
May 28 15:59:12.951297 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.951268    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5dd8s"
May 28 15:59:12.952597 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.952574    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3d8296a6-b347-4d39-a758-d049ee69ef30-hosts-file\") pod \"node-resolver-ztbwc\" (UID: \"3d8296a6-b347-4d39-a758-d049ee69ef30\") " pod="openshift-dns/node-resolver-ztbwc"
May 28 15:59:12.952703 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.952643    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d8296a6-b347-4d39-a758-d049ee69ef30-tmp-dir\") pod \"node-resolver-ztbwc\" (UID: \"3d8296a6-b347-4d39-a758-d049ee69ef30\") " pod="openshift-dns/node-resolver-ztbwc"
May 28 15:59:12.952703 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.952686    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhwfz\" (UniqueName: \"kubernetes.io/projected/bea98d0f-e54c-4da9-a207-f692c8781bea-kube-api-access-dhwfz\") pod \"network-metrics-daemon-69vjf\" (UID: \"bea98d0f-e54c-4da9-a207-f692c8781bea\") " pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:12.952823 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.952711    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47886\" (UniqueName: \"kubernetes.io/projected/c73529ee-0f31-4285-85c2-51cdfd913e80-kube-api-access-47886\") pod \"network-check-target-lw6n8\" (UID: \"c73529ee-0f31-4285-85c2-51cdfd913e80\") " pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:12.952823 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.952733    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-etc-systemd\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:12.952823 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.952756    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2c3786be-4e2a-4969-8c45-45c3b356bc48-agent-certs\") pod \"konnectivity-agent-vj78z\" (UID: \"2c3786be-4e2a-4969-8c45-45c3b356bc48\") " pod="kube-system/konnectivity-agent-vj78z"
May 28 15:59:12.952823 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.952785    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2c3786be-4e2a-4969-8c45-45c3b356bc48-konnectivity-ca\") pod \"konnectivity-agent-vj78z\" (UID: \"2c3786be-4e2a-4969-8c45-45c3b356bc48\") " pod="kube-system/konnectivity-agent-vj78z"
May 28 15:59:12.952823 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.952811    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c4578fd3-b4bc-4b10-85df-d6f40b590795-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rjwv8\" (UID: \"c4578fd3-b4bc-4b10-85df-d6f40b590795\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8"
May 28 15:59:12.953039 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.952839    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs\") pod \"network-metrics-daemon-69vjf\" (UID: \"bea98d0f-e54c-4da9-a207-f692c8781bea\") " pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:12.953039 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.952886    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-run\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:12.953039 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.952914    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/263ec2d4-e89b-48d8-9279-d702116092ad-tmp\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:12.953039 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.952934    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ef8f4bc2-1970-4531-9488-ec899cdc5f35-host-slash\") pod \"iptables-alerter-8krdd\" (UID: \"ef8f4bc2-1970-4531-9488-ec899cdc5f35\") " pod="openshift-network-operator/iptables-alerter-8krdd"
May 28 15:59:12.953039 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.952970    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c4578fd3-b4bc-4b10-85df-d6f40b590795-etc-selinux\") pod \"aws-ebs-csi-driver-node-rjwv8\" (UID: \"c4578fd3-b4bc-4b10-85df-d6f40b590795\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8"
May 28 15:59:12.953039 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.952990    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-sys\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:12.953039 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.953008    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-host\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:12.953370 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.953043    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c4578fd3-b4bc-4b10-85df-d6f40b590795-device-dir\") pod \"aws-ebs-csi-driver-node-rjwv8\" (UID: \"c4578fd3-b4bc-4b10-85df-d6f40b590795\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8"
May 28 15:59:12.953370 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.953098    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-etc-modprobe-d\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:12.953370 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.953116    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-etc-sysconfig\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:12.953370 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.953149    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-lib-modules\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:12.953370 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.953170    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-var-lib-kubelet\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:12.953370 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.953186    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 28 15:59:12.953370 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.953193    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ckjh\" (UniqueName: \"kubernetes.io/projected/ef8f4bc2-1970-4531-9488-ec899cdc5f35-kube-api-access-8ckjh\") pod \"iptables-alerter-8krdd\" (UID: \"ef8f4bc2-1970-4531-9488-ec899cdc5f35\") " pod="openshift-network-operator/iptables-alerter-8krdd"
May 28 15:59:12.953370 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.953214    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-etc-sysctl-d\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:12.953370 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.953237    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ef8f4bc2-1970-4531-9488-ec899cdc5f35-iptables-alerter-script\") pod \"iptables-alerter-8krdd\" (UID: \"ef8f4bc2-1970-4531-9488-ec899cdc5f35\") " pod="openshift-network-operator/iptables-alerter-8krdd"
May 28 15:59:12.953370 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.953247    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 28 15:59:12.953370 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.953259    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjb84\" (UniqueName: \"kubernetes.io/projected/c4578fd3-b4bc-4b10-85df-d6f40b590795-kube-api-access-rjb84\") pod \"aws-ebs-csi-driver-node-rjwv8\" (UID: \"c4578fd3-b4bc-4b10-85df-d6f40b590795\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8"
May 28 15:59:12.953370 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.953254    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-27 15:54:11 +0000 UTC" deadline="2027-11-17 13:07:19.849445852 +0000 UTC"
May 28 15:59:12.953370 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.953282    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-etc-kubernetes\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:12.953370 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.953296    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12909h8m6.896165808s"
May 28 15:59:12.953370 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.953312    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/263ec2d4-e89b-48d8-9279-d702116092ad-etc-tuned\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:12.954005 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.953382    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c4578fd3-b4bc-4b10-85df-d6f40b590795-registration-dir\") pod \"aws-ebs-csi-driver-node-rjwv8\" (UID: \"c4578fd3-b4bc-4b10-85df-d6f40b590795\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8"
May 28 15:59:12.954005 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.953426    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c4578fd3-b4bc-4b10-85df-d6f40b590795-sys-fs\") pod \"aws-ebs-csi-driver-node-rjwv8\" (UID: \"c4578fd3-b4bc-4b10-85df-d6f40b590795\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8"
May 28 15:59:12.954005 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.953456    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l46c7\" (UniqueName: \"kubernetes.io/projected/3d8296a6-b347-4d39-a758-d049ee69ef30-kube-api-access-l46c7\") pod \"node-resolver-ztbwc\" (UID: \"3d8296a6-b347-4d39-a758-d049ee69ef30\") " pod="openshift-dns/node-resolver-ztbwc"
May 28 15:59:12.954005 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.953506    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-etc-sysctl-conf\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:12.954005 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.953530    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkcww\" (UniqueName: \"kubernetes.io/projected/263ec2d4-e89b-48d8-9279-d702116092ad-kube-api-access-mkcww\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:12.954005 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.953550    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c4578fd3-b4bc-4b10-85df-d6f40b590795-socket-dir\") pod \"aws-ebs-csi-driver-node-rjwv8\" (UID: \"c4578fd3-b4bc-4b10-85df-d6f40b590795\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8"
May 28 15:59:12.954005 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.953806    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fgmnc"
May 28 15:59:12.954317 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.954133    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 28 15:59:12.954590 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.954567    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-95ph8\""
May 28 15:59:12.954656 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.954642    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 28 15:59:12.954756 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.954740    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 28 15:59:12.955304 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.955282    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 28 15:59:12.957256 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.956631    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-b46fv\""
May 28 15:59:12.957743 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.957723    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 28 15:59:12.958012 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.957992    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-b2h8l\""
May 28 15:59:12.958276 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.958214    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 28 15:59:12.958276 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.958257    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 28 15:59:12.958904 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.958872    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 28 15:59:12.959000 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.958966    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 28 15:59:12.959055 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.958992    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-m8b2m"
May 28 15:59:12.959102 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.959065    2559 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 15:59:12.960964 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.960946    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 28 15:59:12.961192 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.961175    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-x5lrk\""
May 28 15:59:12.961408 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.961394    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:12.963499 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.963479    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 28 15:59:12.963584 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.963481    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 28 15:59:12.963584 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.963524    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-45n7z\""
May 28 15:59:12.963694 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.963668    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 28 15:59:12.963748 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.963716    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 28 15:59:12.963798 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.963788    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 28 15:59:12.963798 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.963793    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 28 15:59:12.965104 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.965089    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"]
May 28 15:59:12.966361 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.966339    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 15:59:12.966450 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.966438    2559 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal"
May 28 15:59:12.974705 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.974684    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal"]
May 28 15:59:12.974821 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.974804    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 15:59:12.984183 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.984159    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fg96h"
May 28 15:59:12.990212 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:12.990192    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fg96h"
May 28 15:59:13.048811 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.048782    2559 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 28 15:59:13.051898 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:13.051865    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e035ff9b129e563067f987de3c32004.slice/crio-7a97ce5dcf9e6cf6ef3f2880539717022056dce4a775137ad2f603945351b99a WatchSource:0}: Error finding container 7a97ce5dcf9e6cf6ef3f2880539717022056dce4a775137ad2f603945351b99a: Status 404 returned error can't find the container with id 7a97ce5dcf9e6cf6ef3f2880539717022056dce4a775137ad2f603945351b99a
May 28 15:59:13.052158 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:13.052140    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6db7b9c4232c4daf9a5a539bc775452b.slice/crio-cac0dda0fa0fa97a5252f81bbcfe9df7f90a10549a2bd8f0e765ccf069a26b96 WatchSource:0}: Error finding container cac0dda0fa0fa97a5252f81bbcfe9df7f90a10549a2bd8f0e765ccf069a26b96: Status 404 returned error can't find the container with id cac0dda0fa0fa97a5252f81bbcfe9df7f90a10549a2bd8f0e765ccf069a26b96
May 28 15:59:13.053796 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.053754    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-etc-kubernetes\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.053796 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.053802    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/263ec2d4-e89b-48d8-9279-d702116092ad-etc-tuned\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.053962 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.053828    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fd2dd070-87e3-4cda-8fe0-1635aeb20687-host\") pod \"node-ca-5dd8s\" (UID: \"fd2dd070-87e3-4cda-8fe0-1635aeb20687\") " pod="openshift-image-registry/node-ca-5dd8s"
May 28 15:59:13.053962 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.053855    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x956q\" (UniqueName: \"kubernetes.io/projected/fd2dd070-87e3-4cda-8fe0-1635aeb20687-kube-api-access-x956q\") pod \"node-ca-5dd8s\" (UID: \"fd2dd070-87e3-4cda-8fe0-1635aeb20687\") " pod="openshift-image-registry/node-ca-5dd8s"
May 28 15:59:13.053962 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.053873    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht648\" (UniqueName: \"kubernetes.io/projected/bdd26bcc-76e5-4f92-812a-d7d05a31e959-kube-api-access-ht648\") pod \"multus-additional-cni-plugins-fgmnc\" (UID: \"bdd26bcc-76e5-4f92-812a-d7d05a31e959\") " pod="openshift-multus/multus-additional-cni-plugins-fgmnc"
May 28 15:59:13.053962 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.053890    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l46c7\" (UniqueName: \"kubernetes.io/projected/3d8296a6-b347-4d39-a758-d049ee69ef30-kube-api-access-l46c7\") pod \"node-resolver-ztbwc\" (UID: \"3d8296a6-b347-4d39-a758-d049ee69ef30\") " pod="openshift-dns/node-resolver-ztbwc"
May 28 15:59:13.053962 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.053891    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-etc-kubernetes\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.053962 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.053909    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-etc-sysctl-conf\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.053962 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.053952    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mkcww\" (UniqueName: \"kubernetes.io/projected/263ec2d4-e89b-48d8-9279-d702116092ad-kube-api-access-mkcww\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.054254 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.053975    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c4578fd3-b4bc-4b10-85df-d6f40b590795-socket-dir\") pod \"aws-ebs-csi-driver-node-rjwv8\" (UID: \"c4578fd3-b4bc-4b10-85df-d6f40b590795\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8"
May 28 15:59:13.054254 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.053998    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-hostroot\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.054254 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.054026    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d8296a6-b347-4d39-a758-d049ee69ef30-tmp-dir\") pod \"node-resolver-ztbwc\" (UID: \"3d8296a6-b347-4d39-a758-d049ee69ef30\") " pod="openshift-dns/node-resolver-ztbwc"
May 28 15:59:13.054254 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.054054    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-47886\" (UniqueName: \"kubernetes.io/projected/c73529ee-0f31-4285-85c2-51cdfd913e80-kube-api-access-47886\") pod \"network-check-target-lw6n8\" (UID: \"c73529ee-0f31-4285-85c2-51cdfd913e80\") " pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:13.054254 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.054080    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2c3786be-4e2a-4969-8c45-45c3b356bc48-agent-certs\") pod \"konnectivity-agent-vj78z\" (UID: \"2c3786be-4e2a-4969-8c45-45c3b356bc48\") " pod="kube-system/konnectivity-agent-vj78z"
May 28 15:59:13.054254 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.054051    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-etc-sysctl-conf\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.054254 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.054132    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-log-socket\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.054254 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.054157    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c4578fd3-b4bc-4b10-85df-d6f40b590795-socket-dir\") pod \"aws-ebs-csi-driver-node-rjwv8\" (UID: \"c4578fd3-b4bc-4b10-85df-d6f40b590795\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8"
May 28 15:59:13.054254 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.054164    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-host-cni-netd\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.054254 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.054214    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fd6dd018-b7d1-426f-ae3f-4d10ac882495-ovnkube-script-lib\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.054620 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.054287    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-multus-cni-dir\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.054620 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.054293    2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 28 15:59:13.054620 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.054310    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-cnibin\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.054620 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.054352    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c4578fd3-b4bc-4b10-85df-d6f40b590795-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rjwv8\" (UID: \"c4578fd3-b4bc-4b10-85df-d6f40b590795\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8"
May 28 15:59:13.054788 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.054690    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-systemd-units\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.055417 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055109    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fd6dd018-b7d1-426f-ae3f-4d10ac882495-env-overrides\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.055417 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055144    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-host-run-k8s-cni-cncf-io\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.055417 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055177    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-etc-kubernetes\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.055417 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055215    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/263ec2d4-e89b-48d8-9279-d702116092ad-tmp\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.055417 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.054372    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d8296a6-b347-4d39-a758-d049ee69ef30-tmp-dir\") pod \"node-resolver-ztbwc\" (UID: \"3d8296a6-b347-4d39-a758-d049ee69ef30\") " pod="openshift-dns/node-resolver-ztbwc"
May 28 15:59:13.055417 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055262    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ef8f4bc2-1970-4531-9488-ec899cdc5f35-host-slash\") pod \"iptables-alerter-8krdd\" (UID: \"ef8f4bc2-1970-4531-9488-ec899cdc5f35\") " pod="openshift-network-operator/iptables-alerter-8krdd"
May 28 15:59:13.055417 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055295    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c4578fd3-b4bc-4b10-85df-d6f40b590795-etc-selinux\") pod \"aws-ebs-csi-driver-node-rjwv8\" (UID: \"c4578fd3-b4bc-4b10-85df-d6f40b590795\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8"
May 28 15:59:13.055417 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055320    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-node-log\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.055417 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055368    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-cni-binary-copy\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.055417 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055397    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-host-var-lib-cni-multus\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.055848 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055430    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-sys\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.055848 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055461    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-host\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.055848 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055494    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c4578fd3-b4bc-4b10-85df-d6f40b590795-device-dir\") pod \"aws-ebs-csi-driver-node-rjwv8\" (UID: \"c4578fd3-b4bc-4b10-85df-d6f40b590795\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8"
May 28 15:59:13.055848 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055525    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-host-run-ovn-kubernetes\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.055848 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055552    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-host-cni-bin\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.055848 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055582    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fd6dd018-b7d1-426f-ae3f-4d10ac882495-ovnkube-config\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.055848 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055619    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bdd26bcc-76e5-4f92-812a-d7d05a31e959-cnibin\") pod \"multus-additional-cni-plugins-fgmnc\" (UID: \"bdd26bcc-76e5-4f92-812a-d7d05a31e959\") " pod="openshift-multus/multus-additional-cni-plugins-fgmnc"
May 28 15:59:13.055848 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055647    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-system-cni-dir\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.055848 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055679    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-etc-sysconfig\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.055848 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055708    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-var-lib-kubelet\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.055848 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055761    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-run-openvswitch\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.055848 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055804    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-multus-socket-dir-parent\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.055848 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055812    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ef8f4bc2-1970-4531-9488-ec899cdc5f35-host-slash\") pod \"iptables-alerter-8krdd\" (UID: \"ef8f4bc2-1970-4531-9488-ec899cdc5f35\") " pod="openshift-network-operator/iptables-alerter-8krdd"
May 28 15:59:13.055848 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055839    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-host-run-netns\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.056492 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055876    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rjb84\" (UniqueName: \"kubernetes.io/projected/c4578fd3-b4bc-4b10-85df-d6f40b590795-kube-api-access-rjb84\") pod \"aws-ebs-csi-driver-node-rjwv8\" (UID: \"c4578fd3-b4bc-4b10-85df-d6f40b590795\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8"
May 28 15:59:13.056492 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055909    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8j6s\" (UniqueName: \"kubernetes.io/projected/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-kube-api-access-w8j6s\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.056492 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055917    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-sys\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.056492 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055922    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-var-lib-kubelet\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.056492 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055937    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c4578fd3-b4bc-4b10-85df-d6f40b590795-registration-dir\") pod \"aws-ebs-csi-driver-node-rjwv8\" (UID: \"c4578fd3-b4bc-4b10-85df-d6f40b590795\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8"
May 28 15:59:13.056492 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055764    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c4578fd3-b4bc-4b10-85df-d6f40b590795-etc-selinux\") pod \"aws-ebs-csi-driver-node-rjwv8\" (UID: \"c4578fd3-b4bc-4b10-85df-d6f40b590795\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8"
May 28 15:59:13.056492 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.055988    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c4578fd3-b4bc-4b10-85df-d6f40b590795-sys-fs\") pod \"aws-ebs-csi-driver-node-rjwv8\" (UID: \"c4578fd3-b4bc-4b10-85df-d6f40b590795\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8"
May 28 15:59:13.056492 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.056022    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fd6dd018-b7d1-426f-ae3f-4d10ac882495-ovn-node-metrics-cert\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.056492 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.056060    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bdd26bcc-76e5-4f92-812a-d7d05a31e959-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fgmnc\" (UID: \"bdd26bcc-76e5-4f92-812a-d7d05a31e959\") " pod="openshift-multus/multus-additional-cni-plugins-fgmnc"
May 28 15:59:13.056492 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.056081    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c4578fd3-b4bc-4b10-85df-d6f40b590795-registration-dir\") pod \"aws-ebs-csi-driver-node-rjwv8\" (UID: \"c4578fd3-b4bc-4b10-85df-d6f40b590795\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8"
May 28 15:59:13.056492 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.056121    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-etc-sysconfig\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.056492 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.056163    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-host-run-netns\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.056492 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.056199    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-os-release\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.056492 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.056244    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3d8296a6-b347-4d39-a758-d049ee69ef30-hosts-file\") pod \"node-resolver-ztbwc\" (UID: \"3d8296a6-b347-4d39-a758-d049ee69ef30\") " pod="openshift-dns/node-resolver-ztbwc"
May 28 15:59:13.056492 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.056283    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dhwfz\" (UniqueName: \"kubernetes.io/projected/bea98d0f-e54c-4da9-a207-f692c8781bea-kube-api-access-dhwfz\") pod \"network-metrics-daemon-69vjf\" (UID: \"bea98d0f-e54c-4da9-a207-f692c8781bea\") " pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:13.056492 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.056313    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-etc-systemd\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.056492 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.056286    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c4578fd3-b4bc-4b10-85df-d6f40b590795-device-dir\") pod \"aws-ebs-csi-driver-node-rjwv8\" (UID: \"c4578fd3-b4bc-4b10-85df-d6f40b590795\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8"
May 28 15:59:13.057275 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.056531    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-run-systemd\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.057275 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.056569    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bdd26bcc-76e5-4f92-812a-d7d05a31e959-os-release\") pod \"multus-additional-cni-plugins-fgmnc\" (UID: \"bdd26bcc-76e5-4f92-812a-d7d05a31e959\") " pod="openshift-multus/multus-additional-cni-plugins-fgmnc"
May 28 15:59:13.057275 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.056602    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-host-var-lib-cni-bin\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.057275 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.056639    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-host-run-multus-certs\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.057275 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.056684    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2c3786be-4e2a-4969-8c45-45c3b356bc48-konnectivity-ca\") pod \"konnectivity-agent-vj78z\" (UID: \"2c3786be-4e2a-4969-8c45-45c3b356bc48\") " pod="kube-system/konnectivity-agent-vj78z"
May 28 15:59:13.057275 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.056717    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.057275 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.056953    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c4578fd3-b4bc-4b10-85df-d6f40b590795-sys-fs\") pod \"aws-ebs-csi-driver-node-rjwv8\" (UID: \"c4578fd3-b4bc-4b10-85df-d6f40b590795\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8"
May 28 15:59:13.057275 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.056983    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs\") pod \"network-metrics-daemon-69vjf\" (UID: \"bea98d0f-e54c-4da9-a207-f692c8781bea\") " pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:13.057275 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.057012    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-run\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.057275 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.057031    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3d8296a6-b347-4d39-a758-d049ee69ef30-hosts-file\") pod \"node-resolver-ztbwc\" (UID: \"3d8296a6-b347-4d39-a758-d049ee69ef30\") " pod="openshift-dns/node-resolver-ztbwc"
May 28 15:59:13.057275 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.057044    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6fg9\" (UniqueName: \"kubernetes.io/projected/fd6dd018-b7d1-426f-ae3f-4d10ac882495-kube-api-access-m6fg9\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.057275 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.057075    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bdd26bcc-76e5-4f92-812a-d7d05a31e959-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fgmnc\" (UID: \"bdd26bcc-76e5-4f92-812a-d7d05a31e959\") " pod="openshift-multus/multus-additional-cni-plugins-fgmnc"
May 28 15:59:13.057275 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.057100    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-etc-systemd\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.057865 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.057366    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-host-var-lib-kubelet\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.057865 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.057399    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-host-kubelet\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.057865 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.057422    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-etc-openvswitch\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.057865 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.057453    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-etc-modprobe-d\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.057865 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.057487    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-lib-modules\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.057865 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.057551    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8ckjh\" (UniqueName: \"kubernetes.io/projected/ef8f4bc2-1970-4531-9488-ec899cdc5f35-kube-api-access-8ckjh\") pod \"iptables-alerter-8krdd\" (UID: \"ef8f4bc2-1970-4531-9488-ec899cdc5f35\") " pod="openshift-network-operator/iptables-alerter-8krdd"
May 28 15:59:13.057865 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.057553    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-host\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.057865 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:13.057570    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:59:13.057865 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.057594    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-host-slash\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.057865 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.057605    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-run\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.057865 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.057632    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-var-lib-openvswitch\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.057865 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:13.057658    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs podName:bea98d0f-e54c-4da9-a207-f692c8781bea nodeName:}" failed. No retries permitted until 2026-05-28 15:59:13.557624169 +0000 UTC m=+2.117267647 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs") pod "network-metrics-daemon-69vjf" (UID: "bea98d0f-e54c-4da9-a207-f692c8781bea") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:59:13.057865 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.057485    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2c3786be-4e2a-4969-8c45-45c3b356bc48-konnectivity-ca\") pod \"konnectivity-agent-vj78z\" (UID: \"2c3786be-4e2a-4969-8c45-45c3b356bc48\") " pod="kube-system/konnectivity-agent-vj78z"
May 28 15:59:13.058437 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.057997    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-etc-modprobe-d\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.058437 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.058078    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-lib-modules\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.058437 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.058085    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bdd26bcc-76e5-4f92-812a-d7d05a31e959-cni-binary-copy\") pod \"multus-additional-cni-plugins-fgmnc\" (UID: \"bdd26bcc-76e5-4f92-812a-d7d05a31e959\") " pod="openshift-multus/multus-additional-cni-plugins-fgmnc"
May 28 15:59:13.058437 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.058141    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-etc-sysctl-d\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.058437 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.058268    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ef8f4bc2-1970-4531-9488-ec899cdc5f35-iptables-alerter-script\") pod \"iptables-alerter-8krdd\" (UID: \"ef8f4bc2-1970-4531-9488-ec899cdc5f35\") " pod="openshift-network-operator/iptables-alerter-8krdd"
May 28 15:59:13.058437 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.058315    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-run-ovn\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.058437 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.058372    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fd2dd070-87e3-4cda-8fe0-1635aeb20687-serviceca\") pod \"node-ca-5dd8s\" (UID: \"fd2dd070-87e3-4cda-8fe0-1635aeb20687\") " pod="openshift-image-registry/node-ca-5dd8s"
May 28 15:59:13.058848 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.058469    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bdd26bcc-76e5-4f92-812a-d7d05a31e959-system-cni-dir\") pod \"multus-additional-cni-plugins-fgmnc\" (UID: \"bdd26bcc-76e5-4f92-812a-d7d05a31e959\") " pod="openshift-multus/multus-additional-cni-plugins-fgmnc"
May 28 15:59:13.058848 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.058587    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/263ec2d4-e89b-48d8-9279-d702116092ad-etc-sysctl-d\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.058943 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.058904    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ef8f4bc2-1970-4531-9488-ec899cdc5f35-iptables-alerter-script\") pod \"iptables-alerter-8krdd\" (UID: \"ef8f4bc2-1970-4531-9488-ec899cdc5f35\") " pod="openshift-network-operator/iptables-alerter-8krdd"
May 28 15:59:13.058993 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.058973    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bdd26bcc-76e5-4f92-812a-d7d05a31e959-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fgmnc\" (UID: \"bdd26bcc-76e5-4f92-812a-d7d05a31e959\") " pod="openshift-multus/multus-additional-cni-plugins-fgmnc"
May 28 15:59:13.059046 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.059010    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-multus-conf-dir\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.059098 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.059043    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-multus-daemon-config\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.059146 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.054980    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c4578fd3-b4bc-4b10-85df-d6f40b590795-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rjwv8\" (UID: \"c4578fd3-b4bc-4b10-85df-d6f40b590795\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8"
May 28 15:59:13.061031 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.059926    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2c3786be-4e2a-4969-8c45-45c3b356bc48-agent-certs\") pod \"konnectivity-agent-vj78z\" (UID: \"2c3786be-4e2a-4969-8c45-45c3b356bc48\") " pod="kube-system/konnectivity-agent-vj78z"
May 28 15:59:13.061031 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.060717    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/263ec2d4-e89b-48d8-9279-d702116092ad-etc-tuned\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.061263 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.061175    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/263ec2d4-e89b-48d8-9279-d702116092ad-tmp\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.061396 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.061356    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 15:59:13.063876 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:13.063753    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 15:59:13.063876 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:13.063774    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 15:59:13.063876 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:13.063788    2559 projected.go:194] Error preparing data for projected volume kube-api-access-47886 for pod openshift-network-diagnostics/network-check-target-lw6n8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:59:13.063876 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:13.063846    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c73529ee-0f31-4285-85c2-51cdfd913e80-kube-api-access-47886 podName:c73529ee-0f31-4285-85c2-51cdfd913e80 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:13.56382663 +0000 UTC m=+2.123470132 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-47886" (UniqueName: "kubernetes.io/projected/c73529ee-0f31-4285-85c2-51cdfd913e80-kube-api-access-47886") pod "network-check-target-lw6n8" (UID: "c73529ee-0f31-4285-85c2-51cdfd913e80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:59:13.065104 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.065078    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l46c7\" (UniqueName: \"kubernetes.io/projected/3d8296a6-b347-4d39-a758-d049ee69ef30-kube-api-access-l46c7\") pod \"node-resolver-ztbwc\" (UID: \"3d8296a6-b347-4d39-a758-d049ee69ef30\") " pod="openshift-dns/node-resolver-ztbwc"
May 28 15:59:13.065597 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.065575    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkcww\" (UniqueName: \"kubernetes.io/projected/263ec2d4-e89b-48d8-9279-d702116092ad-kube-api-access-mkcww\") pod \"tuned-56thk\" (UID: \"263ec2d4-e89b-48d8-9279-d702116092ad\") " pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.065902 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.065878    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjb84\" (UniqueName: \"kubernetes.io/projected/c4578fd3-b4bc-4b10-85df-d6f40b590795-kube-api-access-rjb84\") pod \"aws-ebs-csi-driver-node-rjwv8\" (UID: \"c4578fd3-b4bc-4b10-85df-d6f40b590795\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8"
May 28 15:59:13.067192 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.067172    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhwfz\" (UniqueName: \"kubernetes.io/projected/bea98d0f-e54c-4da9-a207-f692c8781bea-kube-api-access-dhwfz\") pod \"network-metrics-daemon-69vjf\" (UID: \"bea98d0f-e54c-4da9-a207-f692c8781bea\") " pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:13.067260 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.067213    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ckjh\" (UniqueName: \"kubernetes.io/projected/ef8f4bc2-1970-4531-9488-ec899cdc5f35-kube-api-access-8ckjh\") pod \"iptables-alerter-8krdd\" (UID: \"ef8f4bc2-1970-4531-9488-ec899cdc5f35\") " pod="openshift-network-operator/iptables-alerter-8krdd"
May 28 15:59:13.071805 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.071770    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal" event={"ID":"6e035ff9b129e563067f987de3c32004","Type":"ContainerStarted","Data":"7a97ce5dcf9e6cf6ef3f2880539717022056dce4a775137ad2f603945351b99a"}
May 28 15:59:13.072759 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.072739    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal" event={"ID":"6db7b9c4232c4daf9a5a539bc775452b","Type":"ContainerStarted","Data":"cac0dda0fa0fa97a5252f81bbcfe9df7f90a10549a2bd8f0e765ccf069a26b96"}
May 28 15:59:13.160010 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.159927    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-host-slash\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.160010 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.159959    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-var-lib-openvswitch\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.160010 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160002    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bdd26bcc-76e5-4f92-812a-d7d05a31e959-cni-binary-copy\") pod \"multus-additional-cni-plugins-fgmnc\" (UID: \"bdd26bcc-76e5-4f92-812a-d7d05a31e959\") " pod="openshift-multus/multus-additional-cni-plugins-fgmnc"
May 28 15:59:13.160243 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160021    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-run-ovn\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.160243 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160036    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fd2dd070-87e3-4cda-8fe0-1635aeb20687-serviceca\") pod \"node-ca-5dd8s\" (UID: \"fd2dd070-87e3-4cda-8fe0-1635aeb20687\") " pod="openshift-image-registry/node-ca-5dd8s"
May 28 15:59:13.160243 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160041    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-var-lib-openvswitch\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.160243 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160041    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-host-slash\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.160243 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160052    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bdd26bcc-76e5-4f92-812a-d7d05a31e959-system-cni-dir\") pod \"multus-additional-cni-plugins-fgmnc\" (UID: \"bdd26bcc-76e5-4f92-812a-d7d05a31e959\") " pod="openshift-multus/multus-additional-cni-plugins-fgmnc"
May 28 15:59:13.160243 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160085    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bdd26bcc-76e5-4f92-812a-d7d05a31e959-system-cni-dir\") pod \"multus-additional-cni-plugins-fgmnc\" (UID: \"bdd26bcc-76e5-4f92-812a-d7d05a31e959\") " pod="openshift-multus/multus-additional-cni-plugins-fgmnc"
May 28 15:59:13.160243 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160110    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bdd26bcc-76e5-4f92-812a-d7d05a31e959-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fgmnc\" (UID: \"bdd26bcc-76e5-4f92-812a-d7d05a31e959\") " pod="openshift-multus/multus-additional-cni-plugins-fgmnc"
May 28 15:59:13.160243 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160129    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-multus-conf-dir\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.160243 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160145    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-multus-daemon-config\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.160243 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160075    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-run-ovn\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.160243 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160188    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-multus-conf-dir\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.160243 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160217    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fd2dd070-87e3-4cda-8fe0-1635aeb20687-host\") pod \"node-ca-5dd8s\" (UID: \"fd2dd070-87e3-4cda-8fe0-1635aeb20687\") " pod="openshift-image-registry/node-ca-5dd8s"
May 28 15:59:13.160831 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160267    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x956q\" (UniqueName: \"kubernetes.io/projected/fd2dd070-87e3-4cda-8fe0-1635aeb20687-kube-api-access-x956q\") pod \"node-ca-5dd8s\" (UID: \"fd2dd070-87e3-4cda-8fe0-1635aeb20687\") " pod="openshift-image-registry/node-ca-5dd8s"
May 28 15:59:13.160831 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160291    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ht648\" (UniqueName: \"kubernetes.io/projected/bdd26bcc-76e5-4f92-812a-d7d05a31e959-kube-api-access-ht648\") pod \"multus-additional-cni-plugins-fgmnc\" (UID: \"bdd26bcc-76e5-4f92-812a-d7d05a31e959\") " pod="openshift-multus/multus-additional-cni-plugins-fgmnc"
May 28 15:59:13.160831 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160320    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-hostroot\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.160831 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160370    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fd2dd070-87e3-4cda-8fe0-1635aeb20687-host\") pod \"node-ca-5dd8s\" (UID: \"fd2dd070-87e3-4cda-8fe0-1635aeb20687\") " pod="openshift-image-registry/node-ca-5dd8s"
May 28 15:59:13.160831 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160387    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-log-socket\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.160831 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160455    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-host-cni-netd\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.160831 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160482    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fd6dd018-b7d1-426f-ae3f-4d10ac882495-ovnkube-script-lib\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.160831 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160513    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-multus-cni-dir\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.160831 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160540    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-cnibin\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.160831 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160557    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fd2dd070-87e3-4cda-8fe0-1635aeb20687-serviceca\") pod \"node-ca-5dd8s\" (UID: \"fd2dd070-87e3-4cda-8fe0-1635aeb20687\") " pod="openshift-image-registry/node-ca-5dd8s"
May 28 15:59:13.160831 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160573    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-host-cni-netd\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.160831 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160587    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-systemd-units\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.160831 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160612    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fd6dd018-b7d1-426f-ae3f-4d10ac882495-env-overrides\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.160831 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160616    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-hostroot\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.160831 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160646    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-host-run-k8s-cni-cncf-io\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.160831 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160671    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-systemd-units\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.160831 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160684    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-etc-kubernetes\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.160831 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160713    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-node-log\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.161615 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160732    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-multus-cni-dir\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.161615 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160736    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-cni-binary-copy\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.161615 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160755    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-host-run-k8s-cni-cncf-io\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.161615 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160769    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-host-var-lib-cni-multus\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.161615 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160617    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-log-socket\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.161615 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160772    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bdd26bcc-76e5-4f92-812a-d7d05a31e959-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fgmnc\" (UID: \"bdd26bcc-76e5-4f92-812a-d7d05a31e959\") " pod="openshift-multus/multus-additional-cni-plugins-fgmnc"
May 28 15:59:13.161615 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160802    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-host-run-ovn-kubernetes\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.161615 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160829    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-host-cni-bin\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.161615 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160858    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-etc-kubernetes\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.161615 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160854    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fd6dd018-b7d1-426f-ae3f-4d10ac882495-ovnkube-config\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.161615 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160869    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-multus-daemon-config\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.161615 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160891    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bdd26bcc-76e5-4f92-812a-d7d05a31e959-cnibin\") pod \"multus-additional-cni-plugins-fgmnc\" (UID: \"bdd26bcc-76e5-4f92-812a-d7d05a31e959\") " pod="openshift-multus/multus-additional-cni-plugins-fgmnc"
May 28 15:59:13.161615 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160899    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-host-run-ovn-kubernetes\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.161615 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160897    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-host-cni-bin\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.161615 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160929    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bdd26bcc-76e5-4f92-812a-d7d05a31e959-cni-binary-copy\") pod \"multus-additional-cni-plugins-fgmnc\" (UID: \"bdd26bcc-76e5-4f92-812a-d7d05a31e959\") " pod="openshift-multus/multus-additional-cni-plugins-fgmnc"
May 28 15:59:13.161615 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160938    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-system-cni-dir\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.161615 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160944    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bdd26bcc-76e5-4f92-812a-d7d05a31e959-cnibin\") pod \"multus-additional-cni-plugins-fgmnc\" (UID: \"bdd26bcc-76e5-4f92-812a-d7d05a31e959\") " pod="openshift-multus/multus-additional-cni-plugins-fgmnc"
May 28 15:59:13.161615 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160831    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-host-var-lib-cni-multus\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.162161 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160955    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-node-log\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.162161 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.160988    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-system-cni-dir\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.162161 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161003    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-run-openvswitch\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.162161 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161039    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-cnibin\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.162161 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161044    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-multus-socket-dir-parent\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.162161 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161070    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-run-openvswitch\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.162161 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161111    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-host-run-netns\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.162161 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161151    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-host-run-netns\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.162161 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161159    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w8j6s\" (UniqueName: \"kubernetes.io/projected/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-kube-api-access-w8j6s\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.162161 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161170    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-cni-binary-copy\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.162161 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161182    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fd6dd018-b7d1-426f-ae3f-4d10ac882495-ovnkube-script-lib\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.162161 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161168    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-multus-socket-dir-parent\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.162161 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161210    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fd6dd018-b7d1-426f-ae3f-4d10ac882495-ovn-node-metrics-cert\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.162161 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161238    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bdd26bcc-76e5-4f92-812a-d7d05a31e959-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fgmnc\" (UID: \"bdd26bcc-76e5-4f92-812a-d7d05a31e959\") " pod="openshift-multus/multus-additional-cni-plugins-fgmnc"
May 28 15:59:13.162161 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161265    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-host-run-netns\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.162161 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161292    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-os-release\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.162161 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161302    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-host-run-netns\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.162161 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161317    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fd6dd018-b7d1-426f-ae3f-4d10ac882495-env-overrides\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.162710 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161320    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-run-systemd\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.162710 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161376    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-run-systemd\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.162710 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161381    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bdd26bcc-76e5-4f92-812a-d7d05a31e959-os-release\") pod \"multus-additional-cni-plugins-fgmnc\" (UID: \"bdd26bcc-76e5-4f92-812a-d7d05a31e959\") " pod="openshift-multus/multus-additional-cni-plugins-fgmnc"
May 28 15:59:13.162710 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161390    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fd6dd018-b7d1-426f-ae3f-4d10ac882495-ovnkube-config\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.162710 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161394    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-os-release\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.162710 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161409    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-host-var-lib-cni-bin\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.162710 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161432    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-host-var-lib-cni-bin\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.162710 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161464    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-host-run-multus-certs\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.162710 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161491    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.162710 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161448    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bdd26bcc-76e5-4f92-812a-d7d05a31e959-os-release\") pod \"multus-additional-cni-plugins-fgmnc\" (UID: \"bdd26bcc-76e5-4f92-812a-d7d05a31e959\") " pod="openshift-multus/multus-additional-cni-plugins-fgmnc"
May 28 15:59:13.162710 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161540    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m6fg9\" (UniqueName: \"kubernetes.io/projected/fd6dd018-b7d1-426f-ae3f-4d10ac882495-kube-api-access-m6fg9\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.162710 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161548    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.162710 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161564    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-host-run-multus-certs\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.162710 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161570    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bdd26bcc-76e5-4f92-812a-d7d05a31e959-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fgmnc\" (UID: \"bdd26bcc-76e5-4f92-812a-d7d05a31e959\") " pod="openshift-multus/multus-additional-cni-plugins-fgmnc"
May 28 15:59:13.162710 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161603    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-host-var-lib-kubelet\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.162710 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161629    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-host-kubelet\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.162710 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161666    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bdd26bcc-76e5-4f92-812a-d7d05a31e959-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fgmnc\" (UID: \"bdd26bcc-76e5-4f92-812a-d7d05a31e959\") " pod="openshift-multus/multus-additional-cni-plugins-fgmnc"
May 28 15:59:13.163220 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161669    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-host-kubelet\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.163220 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161689    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bdd26bcc-76e5-4f92-812a-d7d05a31e959-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fgmnc\" (UID: \"bdd26bcc-76e5-4f92-812a-d7d05a31e959\") " pod="openshift-multus/multus-additional-cni-plugins-fgmnc"
May 28 15:59:13.163220 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161685    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-host-var-lib-kubelet\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.163220 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161718    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-etc-openvswitch\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.163220 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.161669    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fd6dd018-b7d1-426f-ae3f-4d10ac882495-etc-openvswitch\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.163381 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.163286    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fd6dd018-b7d1-426f-ae3f-4d10ac882495-ovn-node-metrics-cert\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.167912 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.167892    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht648\" (UniqueName: \"kubernetes.io/projected/bdd26bcc-76e5-4f92-812a-d7d05a31e959-kube-api-access-ht648\") pod \"multus-additional-cni-plugins-fgmnc\" (UID: \"bdd26bcc-76e5-4f92-812a-d7d05a31e959\") " pod="openshift-multus/multus-additional-cni-plugins-fgmnc"
May 28 15:59:13.168002 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.167940    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x956q\" (UniqueName: \"kubernetes.io/projected/fd2dd070-87e3-4cda-8fe0-1635aeb20687-kube-api-access-x956q\") pod \"node-ca-5dd8s\" (UID: \"fd2dd070-87e3-4cda-8fe0-1635aeb20687\") " pod="openshift-image-registry/node-ca-5dd8s"
May 28 15:59:13.168049 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.168003    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8j6s\" (UniqueName: \"kubernetes.io/projected/bc5f0404-5bae-450f-92d9-d1dbcd458e1a-kube-api-access-w8j6s\") pod \"multus-m8b2m\" (UID: \"bc5f0404-5bae-450f-92d9-d1dbcd458e1a\") " pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.168252 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.168237    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6fg9\" (UniqueName: \"kubernetes.io/projected/fd6dd018-b7d1-426f-ae3f-4d10ac882495-kube-api-access-m6fg9\") pod \"ovnkube-node-bgmxp\" (UID: \"fd6dd018-b7d1-426f-ae3f-4d10ac882495\") " pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.262392 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.262362    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-56thk"
May 28 15:59:13.268713 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:13.268691    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod263ec2d4_e89b_48d8_9279_d702116092ad.slice/crio-cfb9c3ef2f16cf7d3cd5a96707a0143c4472c789b6ee8b47a74748fd02862ced WatchSource:0}: Error finding container cfb9c3ef2f16cf7d3cd5a96707a0143c4472c789b6ee8b47a74748fd02862ced: Status 404 returned error can't find the container with id cfb9c3ef2f16cf7d3cd5a96707a0143c4472c789b6ee8b47a74748fd02862ced
May 28 15:59:13.274309 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.274292    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ztbwc"
May 28 15:59:13.279616 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:13.279596    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d8296a6_b347_4d39_a758_d049ee69ef30.slice/crio-a9678b5d60ea27b593c2655304e1924582c69b0949672c1b281121e00eabfeb6 WatchSource:0}: Error finding container a9678b5d60ea27b593c2655304e1924582c69b0949672c1b281121e00eabfeb6: Status 404 returned error can't find the container with id a9678b5d60ea27b593c2655304e1924582c69b0949672c1b281121e00eabfeb6
May 28 15:59:13.291950 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.291934    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8krdd"
May 28 15:59:13.295492 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.295471    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vj78z"
May 28 15:59:13.297377 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:13.297355    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef8f4bc2_1970_4531_9488_ec899cdc5f35.slice/crio-1a3a5297d47ce3125d9226e4d99b3d7daf04c4f9dccdb21959ee3c0f9f375336 WatchSource:0}: Error finding container 1a3a5297d47ce3125d9226e4d99b3d7daf04c4f9dccdb21959ee3c0f9f375336: Status 404 returned error can't find the container with id 1a3a5297d47ce3125d9226e4d99b3d7daf04c4f9dccdb21959ee3c0f9f375336
May 28 15:59:13.302054 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:13.302032    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c3786be_4e2a_4969_8c45_45c3b356bc48.slice/crio-5bba53baf3b2439597bdc64e69ebae8f54f956b218987c2080b3b96a63cebd6d WatchSource:0}: Error finding container 5bba53baf3b2439597bdc64e69ebae8f54f956b218987c2080b3b96a63cebd6d: Status 404 returned error can't find the container with id 5bba53baf3b2439597bdc64e69ebae8f54f956b218987c2080b3b96a63cebd6d
May 28 15:59:13.324043 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.324016    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8"
May 28 15:59:13.329509 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.329487    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5dd8s"
May 28 15:59:13.329722 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:13.329695    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4578fd3_b4bc_4b10_85df_d6f40b590795.slice/crio-d3831d1e9313660f58a9e8db2d89753a1895edd604493bc1106ec56ed909c3b4 WatchSource:0}: Error finding container d3831d1e9313660f58a9e8db2d89753a1895edd604493bc1106ec56ed909c3b4: Status 404 returned error can't find the container with id d3831d1e9313660f58a9e8db2d89753a1895edd604493bc1106ec56ed909c3b4
May 28 15:59:13.335144 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:13.334991    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd2dd070_87e3_4cda_8fe0_1635aeb20687.slice/crio-40d65d41a049e83769abe82aa7cac2b998da70a2e0682bd6a727ac71ee1ab280 WatchSource:0}: Error finding container 40d65d41a049e83769abe82aa7cac2b998da70a2e0682bd6a727ac71ee1ab280: Status 404 returned error can't find the container with id 40d65d41a049e83769abe82aa7cac2b998da70a2e0682bd6a727ac71ee1ab280
May 28 15:59:13.363245 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.363216    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fgmnc"
May 28 15:59:13.368786 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:13.368761    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdd26bcc_76e5_4f92_812a_d7d05a31e959.slice/crio-97303ddff9bb4e8ef6a91fab8acf3fdf3b74b7d953ade6154317d09f51dd69d7 WatchSource:0}: Error finding container 97303ddff9bb4e8ef6a91fab8acf3fdf3b74b7d953ade6154317d09f51dd69d7: Status 404 returned error can't find the container with id 97303ddff9bb4e8ef6a91fab8acf3fdf3b74b7d953ade6154317d09f51dd69d7
May 28 15:59:13.369292 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.369269    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-m8b2m"
May 28 15:59:13.375050 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.375026    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:13.375705 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:13.375679    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc5f0404_5bae_450f_92d9_d1dbcd458e1a.slice/crio-b1a68a5f1daf52c0925ed86d4e7a4a9fa3d127d342553d670a6d4925b3c63d38 WatchSource:0}: Error finding container b1a68a5f1daf52c0925ed86d4e7a4a9fa3d127d342553d670a6d4925b3c63d38: Status 404 returned error can't find the container with id b1a68a5f1daf52c0925ed86d4e7a4a9fa3d127d342553d670a6d4925b3c63d38
May 28 15:59:13.383555 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:13.383532    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd6dd018_b7d1_426f_ae3f_4d10ac882495.slice/crio-13313b4dba8161173fe056e918acf302866e6af79e254a4927a3ac52a5b485c5 WatchSource:0}: Error finding container 13313b4dba8161173fe056e918acf302866e6af79e254a4927a3ac52a5b485c5: Status 404 returned error can't find the container with id 13313b4dba8161173fe056e918acf302866e6af79e254a4927a3ac52a5b485c5
May 28 15:59:13.474813 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.474741    2559 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 15:59:13.565126 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.565091    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-47886\" (UniqueName: \"kubernetes.io/projected/c73529ee-0f31-4285-85c2-51cdfd913e80-kube-api-access-47886\") pod \"network-check-target-lw6n8\" (UID: \"c73529ee-0f31-4285-85c2-51cdfd913e80\") " pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:13.565309 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.565246    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs\") pod \"network-metrics-daemon-69vjf\" (UID: \"bea98d0f-e54c-4da9-a207-f692c8781bea\") " pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:13.565309 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:13.565280    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 15:59:13.565309 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:13.565305    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 15:59:13.565497 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:13.565319    2559 projected.go:194] Error preparing data for projected volume kube-api-access-47886 for pod openshift-network-diagnostics/network-check-target-lw6n8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:59:13.565497 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:13.565394    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:59:13.565497 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:13.565400    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c73529ee-0f31-4285-85c2-51cdfd913e80-kube-api-access-47886 podName:c73529ee-0f31-4285-85c2-51cdfd913e80 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:14.56538006 +0000 UTC m=+3.125023539 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-47886" (UniqueName: "kubernetes.io/projected/c73529ee-0f31-4285-85c2-51cdfd913e80-kube-api-access-47886") pod "network-check-target-lw6n8" (UID: "c73529ee-0f31-4285-85c2-51cdfd913e80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:59:13.565497 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:13.565448    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs podName:bea98d0f-e54c-4da9-a207-f692c8781bea nodeName:}" failed. No retries permitted until 2026-05-28 15:59:14.565432413 +0000 UTC m=+3.125075892 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs") pod "network-metrics-daemon-69vjf" (UID: "bea98d0f-e54c-4da9-a207-f692c8781bea") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:59:13.991815 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.991767    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 15:54:12 +0000 UTC" deadline="2027-10-21 22:46:03.541839744 +0000 UTC"
May 28 15:59:13.991815 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:13.991812    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12270h46m49.550031448s"
May 28 15:59:14.091921 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:14.091834    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp" event={"ID":"fd6dd018-b7d1-426f-ae3f-4d10ac882495","Type":"ContainerStarted","Data":"13313b4dba8161173fe056e918acf302866e6af79e254a4927a3ac52a5b485c5"}
May 28 15:59:14.102670 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:14.102613    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-m8b2m" event={"ID":"bc5f0404-5bae-450f-92d9-d1dbcd458e1a","Type":"ContainerStarted","Data":"b1a68a5f1daf52c0925ed86d4e7a4a9fa3d127d342553d670a6d4925b3c63d38"}
May 28 15:59:14.112187 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:14.112152    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vj78z" event={"ID":"2c3786be-4e2a-4969-8c45-45c3b356bc48","Type":"ContainerStarted","Data":"5bba53baf3b2439597bdc64e69ebae8f54f956b218987c2080b3b96a63cebd6d"}
May 28 15:59:14.121571 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:14.121534    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8krdd" event={"ID":"ef8f4bc2-1970-4531-9488-ec899cdc5f35","Type":"ContainerStarted","Data":"1a3a5297d47ce3125d9226e4d99b3d7daf04c4f9dccdb21959ee3c0f9f375336"}
May 28 15:59:14.131984 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:14.131919    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ztbwc" event={"ID":"3d8296a6-b347-4d39-a758-d049ee69ef30","Type":"ContainerStarted","Data":"a9678b5d60ea27b593c2655304e1924582c69b0949672c1b281121e00eabfeb6"}
May 28 15:59:14.136850 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:14.136802    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fgmnc" event={"ID":"bdd26bcc-76e5-4f92-812a-d7d05a31e959","Type":"ContainerStarted","Data":"97303ddff9bb4e8ef6a91fab8acf3fdf3b74b7d953ade6154317d09f51dd69d7"}
May 28 15:59:14.139058 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:14.139034    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5dd8s" event={"ID":"fd2dd070-87e3-4cda-8fe0-1635aeb20687","Type":"ContainerStarted","Data":"40d65d41a049e83769abe82aa7cac2b998da70a2e0682bd6a727ac71ee1ab280"}
May 28 15:59:14.146145 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:14.146119    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8" event={"ID":"c4578fd3-b4bc-4b10-85df-d6f40b590795","Type":"ContainerStarted","Data":"d3831d1e9313660f58a9e8db2d89753a1895edd604493bc1106ec56ed909c3b4"}
May 28 15:59:14.170199 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:14.170158    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-56thk" event={"ID":"263ec2d4-e89b-48d8-9279-d702116092ad","Type":"ContainerStarted","Data":"cfb9c3ef2f16cf7d3cd5a96707a0143c4472c789b6ee8b47a74748fd02862ced"}
May 28 15:59:14.231843 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:14.231813    2559 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 15:59:14.575835 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:14.575783    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs\") pod \"network-metrics-daemon-69vjf\" (UID: \"bea98d0f-e54c-4da9-a207-f692c8781bea\") " pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:14.576027 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:14.575856    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-47886\" (UniqueName: \"kubernetes.io/projected/c73529ee-0f31-4285-85c2-51cdfd913e80-kube-api-access-47886\") pod \"network-check-target-lw6n8\" (UID: \"c73529ee-0f31-4285-85c2-51cdfd913e80\") " pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:14.576027 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:14.576007    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 15:59:14.576027 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:14.576028    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 15:59:14.576210 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:14.576042    2559 projected.go:194] Error preparing data for projected volume kube-api-access-47886 for pod openshift-network-diagnostics/network-check-target-lw6n8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:59:14.576210 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:14.576104    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c73529ee-0f31-4285-85c2-51cdfd913e80-kube-api-access-47886 podName:c73529ee-0f31-4285-85c2-51cdfd913e80 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:16.576084956 +0000 UTC m=+5.135728440 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-47886" (UniqueName: "kubernetes.io/projected/c73529ee-0f31-4285-85c2-51cdfd913e80-kube-api-access-47886") pod "network-check-target-lw6n8" (UID: "c73529ee-0f31-4285-85c2-51cdfd913e80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:59:14.576607 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:14.576586    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:59:14.576711 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:14.576642    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs podName:bea98d0f-e54c-4da9-a207-f692c8781bea nodeName:}" failed. No retries permitted until 2026-05-28 15:59:16.576625994 +0000 UTC m=+5.136269475 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs") pod "network-metrics-daemon-69vjf" (UID: "bea98d0f-e54c-4da9-a207-f692c8781bea") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:59:14.992722 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:14.992673    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 15:54:12 +0000 UTC" deadline="2028-01-01 22:51:16.832558969 +0000 UTC"
May 28 15:59:14.992722 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:14.992720    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13998h52m1.839843128s"
May 28 15:59:15.070155 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:15.070125    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:15.070354 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:15.070251    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lw6n8" podUID="c73529ee-0f31-4285-85c2-51cdfd913e80"
May 28 15:59:15.070738 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:15.070717    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:15.070842 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:15.070822    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69vjf" podUID="bea98d0f-e54c-4da9-a207-f692c8781bea"
May 28 15:59:16.595922 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:16.595883    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs\") pod \"network-metrics-daemon-69vjf\" (UID: \"bea98d0f-e54c-4da9-a207-f692c8781bea\") " pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:16.596515 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:16.595958    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-47886\" (UniqueName: \"kubernetes.io/projected/c73529ee-0f31-4285-85c2-51cdfd913e80-kube-api-access-47886\") pod \"network-check-target-lw6n8\" (UID: \"c73529ee-0f31-4285-85c2-51cdfd913e80\") " pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:16.596515 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:16.596061    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:59:16.596515 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:16.596100    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 15:59:16.596515 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:16.596118    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 15:59:16.596515 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:16.596131    2559 projected.go:194] Error preparing data for projected volume kube-api-access-47886 for pod openshift-network-diagnostics/network-check-target-lw6n8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:59:16.596515 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:16.596143    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs podName:bea98d0f-e54c-4da9-a207-f692c8781bea nodeName:}" failed. No retries permitted until 2026-05-28 15:59:20.59612234 +0000 UTC m=+9.155765831 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs") pod "network-metrics-daemon-69vjf" (UID: "bea98d0f-e54c-4da9-a207-f692c8781bea") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:59:16.596515 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:16.596178    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c73529ee-0f31-4285-85c2-51cdfd913e80-kube-api-access-47886 podName:c73529ee-0f31-4285-85c2-51cdfd913e80 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:20.596162625 +0000 UTC m=+9.155806114 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-47886" (UniqueName: "kubernetes.io/projected/c73529ee-0f31-4285-85c2-51cdfd913e80-kube-api-access-47886") pod "network-check-target-lw6n8" (UID: "c73529ee-0f31-4285-85c2-51cdfd913e80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:59:17.070201 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:17.070167    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:17.070396 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:17.070307    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lw6n8" podUID="c73529ee-0f31-4285-85c2-51cdfd913e80"
May 28 15:59:17.070743 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:17.070720    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:17.070867 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:17.070844    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69vjf" podUID="bea98d0f-e54c-4da9-a207-f692c8781bea"
May 28 15:59:19.069950 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:19.069916    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:19.070434 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:19.070052    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69vjf" podUID="bea98d0f-e54c-4da9-a207-f692c8781bea"
May 28 15:59:19.070434 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:19.070086    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:19.070434 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:19.070175    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lw6n8" podUID="c73529ee-0f31-4285-85c2-51cdfd913e80"
May 28 15:59:20.630557 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:20.630517    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-47886\" (UniqueName: \"kubernetes.io/projected/c73529ee-0f31-4285-85c2-51cdfd913e80-kube-api-access-47886\") pod \"network-check-target-lw6n8\" (UID: \"c73529ee-0f31-4285-85c2-51cdfd913e80\") " pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:20.630992 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:20.630606    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs\") pod \"network-metrics-daemon-69vjf\" (UID: \"bea98d0f-e54c-4da9-a207-f692c8781bea\") " pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:20.630992 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:20.630703    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 15:59:20.630992 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:20.630725    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:59:20.630992 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:20.630730    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 15:59:20.630992 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:20.630746    2559 projected.go:194] Error preparing data for projected volume kube-api-access-47886 for pod openshift-network-diagnostics/network-check-target-lw6n8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:59:20.630992 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:20.630792    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs podName:bea98d0f-e54c-4da9-a207-f692c8781bea nodeName:}" failed. No retries permitted until 2026-05-28 15:59:28.630773037 +0000 UTC m=+17.190416532 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs") pod "network-metrics-daemon-69vjf" (UID: "bea98d0f-e54c-4da9-a207-f692c8781bea") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:59:20.630992 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:20.630814    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c73529ee-0f31-4285-85c2-51cdfd913e80-kube-api-access-47886 podName:c73529ee-0f31-4285-85c2-51cdfd913e80 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:28.630802542 +0000 UTC m=+17.190446031 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-47886" (UniqueName: "kubernetes.io/projected/c73529ee-0f31-4285-85c2-51cdfd913e80-kube-api-access-47886") pod "network-check-target-lw6n8" (UID: "c73529ee-0f31-4285-85c2-51cdfd913e80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:59:21.070005 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:21.069968    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:21.070195 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:21.070104    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lw6n8" podUID="c73529ee-0f31-4285-85c2-51cdfd913e80"
May 28 15:59:21.070267 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:21.070199    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:21.070391 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:21.070316    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69vjf" podUID="bea98d0f-e54c-4da9-a207-f692c8781bea"
May 28 15:59:23.070215 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:23.070181    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:23.070571 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:23.070181    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:23.070571 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:23.070310    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69vjf" podUID="bea98d0f-e54c-4da9-a207-f692c8781bea"
May 28 15:59:23.070571 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:23.070358    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lw6n8" podUID="c73529ee-0f31-4285-85c2-51cdfd913e80"
May 28 15:59:25.069778 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:25.069746    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:25.070221 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:25.069755    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:25.070221 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:25.069878    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69vjf" podUID="bea98d0f-e54c-4da9-a207-f692c8781bea"
May 28 15:59:25.070221 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:25.069932    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lw6n8" podUID="c73529ee-0f31-4285-85c2-51cdfd913e80"
May 28 15:59:27.069961 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:27.069917    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:27.070357 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:27.069915    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:27.070357 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:27.070055    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69vjf" podUID="bea98d0f-e54c-4da9-a207-f692c8781bea"
May 28 15:59:27.070357 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:27.070095    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lw6n8" podUID="c73529ee-0f31-4285-85c2-51cdfd913e80"
May 28 15:59:28.682571 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:28.682536    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-47886\" (UniqueName: \"kubernetes.io/projected/c73529ee-0f31-4285-85c2-51cdfd913e80-kube-api-access-47886\") pod \"network-check-target-lw6n8\" (UID: \"c73529ee-0f31-4285-85c2-51cdfd913e80\") " pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:28.683094 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:28.682612    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs\") pod \"network-metrics-daemon-69vjf\" (UID: \"bea98d0f-e54c-4da9-a207-f692c8781bea\") " pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:28.683094 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:28.682713    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 15:59:28.683094 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:28.682726    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:59:28.683094 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:28.682738    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 15:59:28.683094 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:28.682748    2559 projected.go:194] Error preparing data for projected volume kube-api-access-47886 for pod openshift-network-diagnostics/network-check-target-lw6n8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:59:28.683094 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:28.682796    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs podName:bea98d0f-e54c-4da9-a207-f692c8781bea nodeName:}" failed. No retries permitted until 2026-05-28 15:59:44.682775344 +0000 UTC m=+33.242418838 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs") pod "network-metrics-daemon-69vjf" (UID: "bea98d0f-e54c-4da9-a207-f692c8781bea") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:59:28.683094 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:28.682818    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c73529ee-0f31-4285-85c2-51cdfd913e80-kube-api-access-47886 podName:c73529ee-0f31-4285-85c2-51cdfd913e80 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:44.682806601 +0000 UTC m=+33.242450081 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-47886" (UniqueName: "kubernetes.io/projected/c73529ee-0f31-4285-85c2-51cdfd913e80-kube-api-access-47886") pod "network-check-target-lw6n8" (UID: "c73529ee-0f31-4285-85c2-51cdfd913e80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:59:29.070045 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:29.070008    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:29.070225 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:29.070008    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:29.070225 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:29.070154    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lw6n8" podUID="c73529ee-0f31-4285-85c2-51cdfd913e80"
May 28 15:59:29.070369 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:29.070223    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69vjf" podUID="bea98d0f-e54c-4da9-a207-f692c8781bea"
May 28 15:59:29.204861 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:29.204827    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-bkvg9"]
May 28 15:59:29.212442 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:29.212418    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 15:59:29.212570 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:29.212502    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bkvg9" podUID="369ece78-a49b-484b-8521-d1f4656e1c6e"
May 28 15:59:29.287725 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:29.287690    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/369ece78-a49b-484b-8521-d1f4656e1c6e-original-pull-secret\") pod \"global-pull-secret-syncer-bkvg9\" (UID: \"369ece78-a49b-484b-8521-d1f4656e1c6e\") " pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 15:59:29.287886 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:29.287748    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/369ece78-a49b-484b-8521-d1f4656e1c6e-kubelet-config\") pod \"global-pull-secret-syncer-bkvg9\" (UID: \"369ece78-a49b-484b-8521-d1f4656e1c6e\") " pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 15:59:29.287886 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:29.287772    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/369ece78-a49b-484b-8521-d1f4656e1c6e-dbus\") pod \"global-pull-secret-syncer-bkvg9\" (UID: \"369ece78-a49b-484b-8521-d1f4656e1c6e\") " pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 15:59:29.388481 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:29.388392    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/369ece78-a49b-484b-8521-d1f4656e1c6e-kubelet-config\") pod \"global-pull-secret-syncer-bkvg9\" (UID: \"369ece78-a49b-484b-8521-d1f4656e1c6e\") " pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 15:59:29.388481 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:29.388455    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/369ece78-a49b-484b-8521-d1f4656e1c6e-dbus\") pod \"global-pull-secret-syncer-bkvg9\" (UID: \"369ece78-a49b-484b-8521-d1f4656e1c6e\") " pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 15:59:29.388708 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:29.388496    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/369ece78-a49b-484b-8521-d1f4656e1c6e-kubelet-config\") pod \"global-pull-secret-syncer-bkvg9\" (UID: \"369ece78-a49b-484b-8521-d1f4656e1c6e\") " pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 15:59:29.388708 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:29.388515    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/369ece78-a49b-484b-8521-d1f4656e1c6e-original-pull-secret\") pod \"global-pull-secret-syncer-bkvg9\" (UID: \"369ece78-a49b-484b-8521-d1f4656e1c6e\") " pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 15:59:29.388708 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:29.388631    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 15:59:29.388708 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:29.388687    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/369ece78-a49b-484b-8521-d1f4656e1c6e-dbus\") pod \"global-pull-secret-syncer-bkvg9\" (UID: \"369ece78-a49b-484b-8521-d1f4656e1c6e\") " pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 15:59:29.388708 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:29.388702    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/369ece78-a49b-484b-8521-d1f4656e1c6e-original-pull-secret podName:369ece78-a49b-484b-8521-d1f4656e1c6e nodeName:}" failed. No retries permitted until 2026-05-28 15:59:29.888687303 +0000 UTC m=+18.448330779 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/369ece78-a49b-484b-8521-d1f4656e1c6e-original-pull-secret") pod "global-pull-secret-syncer-bkvg9" (UID: "369ece78-a49b-484b-8521-d1f4656e1c6e") : object "kube-system"/"original-pull-secret" not registered
May 28 15:59:29.891406 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:29.891353    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/369ece78-a49b-484b-8521-d1f4656e1c6e-original-pull-secret\") pod \"global-pull-secret-syncer-bkvg9\" (UID: \"369ece78-a49b-484b-8521-d1f4656e1c6e\") " pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 15:59:29.891835 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:29.891518    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 15:59:29.891835 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:29.891593    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/369ece78-a49b-484b-8521-d1f4656e1c6e-original-pull-secret podName:369ece78-a49b-484b-8521-d1f4656e1c6e nodeName:}" failed. No retries permitted until 2026-05-28 15:59:30.891575322 +0000 UTC m=+19.451218801 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/369ece78-a49b-484b-8521-d1f4656e1c6e-original-pull-secret") pod "global-pull-secret-syncer-bkvg9" (UID: "369ece78-a49b-484b-8521-d1f4656e1c6e") : object "kube-system"/"original-pull-secret" not registered
May 28 15:59:30.898234 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:30.898194    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/369ece78-a49b-484b-8521-d1f4656e1c6e-original-pull-secret\") pod \"global-pull-secret-syncer-bkvg9\" (UID: \"369ece78-a49b-484b-8521-d1f4656e1c6e\") " pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 15:59:30.898675 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:30.898378    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 15:59:30.898675 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:30.898459    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/369ece78-a49b-484b-8521-d1f4656e1c6e-original-pull-secret podName:369ece78-a49b-484b-8521-d1f4656e1c6e nodeName:}" failed. No retries permitted until 2026-05-28 15:59:32.898437369 +0000 UTC m=+21.458080850 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/369ece78-a49b-484b-8521-d1f4656e1c6e-original-pull-secret") pod "global-pull-secret-syncer-bkvg9" (UID: "369ece78-a49b-484b-8521-d1f4656e1c6e") : object "kube-system"/"original-pull-secret" not registered
May 28 15:59:31.069864 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:31.069831    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 15:59:31.070049 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:31.069873    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:31.070049 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:31.069906    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:31.070049 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:31.069998    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69vjf" podUID="bea98d0f-e54c-4da9-a207-f692c8781bea"
May 28 15:59:31.070211 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:31.070135    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lw6n8" podUID="c73529ee-0f31-4285-85c2-51cdfd913e80"
May 28 15:59:31.070266 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:31.070240    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bkvg9" podUID="369ece78-a49b-484b-8521-d1f4656e1c6e"
May 28 15:59:32.912028 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:32.911842    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/369ece78-a49b-484b-8521-d1f4656e1c6e-original-pull-secret\") pod \"global-pull-secret-syncer-bkvg9\" (UID: \"369ece78-a49b-484b-8521-d1f4656e1c6e\") " pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 15:59:32.912485 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:32.911945    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 15:59:32.912485 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:32.912140    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/369ece78-a49b-484b-8521-d1f4656e1c6e-original-pull-secret podName:369ece78-a49b-484b-8521-d1f4656e1c6e nodeName:}" failed. No retries permitted until 2026-05-28 15:59:36.912123222 +0000 UTC m=+25.471766702 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/369ece78-a49b-484b-8521-d1f4656e1c6e-original-pull-secret") pod "global-pull-secret-syncer-bkvg9" (UID: "369ece78-a49b-484b-8521-d1f4656e1c6e") : object "kube-system"/"original-pull-secret" not registered
May 28 15:59:33.069864 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.069829    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:33.069995 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.069829    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:33.069995 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:33.069930    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lw6n8" podUID="c73529ee-0f31-4285-85c2-51cdfd913e80"
May 28 15:59:33.070075 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:33.070015    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69vjf" podUID="bea98d0f-e54c-4da9-a207-f692c8781bea"
May 28 15:59:33.070075 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.069829    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 15:59:33.070133 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:33.070082    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bkvg9" podUID="369ece78-a49b-484b-8521-d1f4656e1c6e"
May 28 15:59:33.214602 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.214519    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5dd8s" event={"ID":"fd2dd070-87e3-4cda-8fe0-1635aeb20687","Type":"ContainerStarted","Data":"a6a87f0712a16d7f3026d9868097a3419fcc56be0d683f96a5a0be3bd0d30042"}
May 28 15:59:33.215990 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.215961    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8" event={"ID":"c4578fd3-b4bc-4b10-85df-d6f40b590795","Type":"ContainerStarted","Data":"50e5df4c836e7d3325503d0e410dd3135720d0f925d87236afa52d701893a185"}
May 28 15:59:33.217276 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.217254    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-56thk" event={"ID":"263ec2d4-e89b-48d8-9279-d702116092ad","Type":"ContainerStarted","Data":"25694784f16d43c72ed77d46daca16ddbd18958821252df3c63be1c3a05ac2e2"}
May 28 15:59:33.218748 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.218715    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal" event={"ID":"6db7b9c4232c4daf9a5a539bc775452b","Type":"ContainerStarted","Data":"5dd85788d6d7085faa2e856927b315398c7092dd083fe150696ae1d66c24b421"}
May 28 15:59:33.220233 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.220203    2559 generic.go:358] "Generic (PLEG): container finished" podID="6e035ff9b129e563067f987de3c32004" containerID="dc913b13d38350b29ebbfbb2f11fe5a21d6aadc77b9c4e233b0dea81478ced80" exitCode=0
May 28 15:59:33.220351 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.220265    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal" event={"ID":"6e035ff9b129e563067f987de3c32004","Type":"ContainerDied","Data":"dc913b13d38350b29ebbfbb2f11fe5a21d6aadc77b9c4e233b0dea81478ced80"}
May 28 15:59:33.223088 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.223045    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp" event={"ID":"fd6dd018-b7d1-426f-ae3f-4d10ac882495","Type":"ContainerStarted","Data":"986101a16b861eaecdfcf6e9ace2caa4fcf4d5f5a00940877ff24c5f93779b62"}
May 28 15:59:33.223178 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.223096    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp" event={"ID":"fd6dd018-b7d1-426f-ae3f-4d10ac882495","Type":"ContainerStarted","Data":"28ae9c2339c5df16bff47b98124af9bc253490fb4ed78ff152a1361127a5a915"}
May 28 15:59:33.223178 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.223112    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp" event={"ID":"fd6dd018-b7d1-426f-ae3f-4d10ac882495","Type":"ContainerStarted","Data":"6cc2e991dad5772f758c39a9f5c511353ed53ed16f0848fdbac5330b4c42e2a3"}
May 28 15:59:33.223178 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.223124    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp" event={"ID":"fd6dd018-b7d1-426f-ae3f-4d10ac882495","Type":"ContainerStarted","Data":"89d14628d7cb4b33cd38d5640fee27963812d0e208ab4b261f238bf71121a9e2"}
May 28 15:59:33.223178 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.223135    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp" event={"ID":"fd6dd018-b7d1-426f-ae3f-4d10ac882495","Type":"ContainerStarted","Data":"33cf68d1e49685c73b53d86613452c65d53790fe0d3a291f0e4bfe4abcd52b4e"}
May 28 15:59:33.223178 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.223147    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp" event={"ID":"fd6dd018-b7d1-426f-ae3f-4d10ac882495","Type":"ContainerStarted","Data":"aac334f19633fd5eb5e790759fb0a88f4dd86733ab335acb2d76492258f4c7b3"}
May 28 15:59:33.224394 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.224372    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-m8b2m" event={"ID":"bc5f0404-5bae-450f-92d9-d1dbcd458e1a","Type":"ContainerStarted","Data":"b21d3159dbc6e2e431cdd878bc6467ab65bb92e8aa460dbf98a00fd771a752d7"}
May 28 15:59:33.225765 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.225742    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vj78z" event={"ID":"2c3786be-4e2a-4969-8c45-45c3b356bc48","Type":"ContainerStarted","Data":"83444e067a660cd5c0e464d334aea9f1f355f5a779592dbf5f9117d6b4cb7d01"}
May 28 15:59:33.226905 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.226868    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5dd8s" podStartSLOduration=2.394464853 podStartE2EDuration="21.226857899s" podCreationTimestamp="2026-05-28 15:59:12 +0000 UTC" firstStartedPulling="2026-05-28 15:59:13.336524267 +0000 UTC m=+1.896167743" lastFinishedPulling="2026-05-28 15:59:32.168917298 +0000 UTC m=+20.728560789" observedRunningTime="2026-05-28 15:59:33.226662682 +0000 UTC m=+21.786306181" watchObservedRunningTime="2026-05-28 15:59:33.226857899 +0000 UTC m=+21.786501396"
May 28 15:59:33.227039 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.227018    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ztbwc" event={"ID":"3d8296a6-b347-4d39-a758-d049ee69ef30","Type":"ContainerStarted","Data":"b5d7c50d4830a71684d49ab6da186a14da0e5aa784aa1e654aac13178c2c5851"}
May 28 15:59:33.228684 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.228646    2559 generic.go:358] "Generic (PLEG): container finished" podID="bdd26bcc-76e5-4f92-812a-d7d05a31e959" containerID="8ded45526861f85c6d29172fffab9b3c93c42caee41ea635cca17eaa5a182b62" exitCode=0
May 28 15:59:33.228776 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.228694    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fgmnc" event={"ID":"bdd26bcc-76e5-4f92-812a-d7d05a31e959","Type":"ContainerDied","Data":"8ded45526861f85c6d29172fffab9b3c93c42caee41ea635cca17eaa5a182b62"}
May 28 15:59:33.249942 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.249894    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-m8b2m" podStartSLOduration=2.36466223 podStartE2EDuration="21.249878192s" podCreationTimestamp="2026-05-28 15:59:12 +0000 UTC" firstStartedPulling="2026-05-28 15:59:13.378805074 +0000 UTC m=+1.938448549" lastFinishedPulling="2026-05-28 15:59:32.264021029 +0000 UTC m=+20.823664511" observedRunningTime="2026-05-28 15:59:33.238619609 +0000 UTC m=+21.798263108" watchObservedRunningTime="2026-05-28 15:59:33.249878192 +0000 UTC m=+21.809521691"
May 28 15:59:33.250661 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.250619    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vj78z" podStartSLOduration=2.2912629620000002 podStartE2EDuration="21.250607907s" podCreationTimestamp="2026-05-28 15:59:12 +0000 UTC" firstStartedPulling="2026-05-28 15:59:13.303452932 +0000 UTC m=+1.863096409" lastFinishedPulling="2026-05-28 15:59:32.262797867 +0000 UTC m=+20.822441354" observedRunningTime="2026-05-28 15:59:33.249710136 +0000 UTC m=+21.809353635" watchObservedRunningTime="2026-05-28 15:59:33.250607907 +0000 UTC m=+21.810251405"
May 28 15:59:33.277213 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.277156    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal" podStartSLOduration=21.27713704 podStartE2EDuration="21.27713704s" podCreationTimestamp="2026-05-28 15:59:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 15:59:33.276613402 +0000 UTC m=+21.836256901" watchObservedRunningTime="2026-05-28 15:59:33.27713704 +0000 UTC m=+21.836780540"
May 28 15:59:33.288213 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.288170    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-56thk" podStartSLOduration=2.342315021 podStartE2EDuration="21.288155726s" podCreationTimestamp="2026-05-28 15:59:12 +0000 UTC" firstStartedPulling="2026-05-28 15:59:13.270198791 +0000 UTC m=+1.829842266" lastFinishedPulling="2026-05-28 15:59:32.21603949 +0000 UTC m=+20.775682971" observedRunningTime="2026-05-28 15:59:33.288122061 +0000 UTC m=+21.847765556" watchObservedRunningTime="2026-05-28 15:59:33.288155726 +0000 UTC m=+21.847799246"
May 28 15:59:33.342445 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.342380    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-ztbwc" podStartSLOduration=2.40753601 podStartE2EDuration="21.342359479s" podCreationTimestamp="2026-05-28 15:59:12 +0000 UTC" firstStartedPulling="2026-05-28 15:59:13.281077422 +0000 UTC m=+1.840720897" lastFinishedPulling="2026-05-28 15:59:32.21590089 +0000 UTC m=+20.775544366" observedRunningTime="2026-05-28 15:59:33.342283559 +0000 UTC m=+21.901927056" watchObservedRunningTime="2026-05-28 15:59:33.342359479 +0000 UTC m=+21.902002980"
May 28 15:59:33.778938 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:33.778737    2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 28 15:59:34.026779 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:34.026682    2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-28T15:59:33.778934704Z","UUID":"20df84b0-9e65-46ea-b11d-aa424679180f","Handler":null,"Name":"","Endpoint":""}
May 28 15:59:34.030085 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:34.030058    2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 28 15:59:34.030224 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:34.030095    2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 28 15:59:34.232512 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:34.232449    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8krdd" event={"ID":"ef8f4bc2-1970-4531-9488-ec899cdc5f35","Type":"ContainerStarted","Data":"024e8e3adb30d780869576cba6f426d152ab5a5ba2cba760afb85d51658db242"}
May 28 15:59:34.234318 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:34.234283    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8" event={"ID":"c4578fd3-b4bc-4b10-85df-d6f40b590795","Type":"ContainerStarted","Data":"f071615603100e04e82a500a37461c790e7d96963ae2b97baf2085dc86ddacb4"}
May 28 15:59:34.236313 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:34.236285    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal" event={"ID":"6e035ff9b129e563067f987de3c32004","Type":"ContainerStarted","Data":"db225db292a8d38ccf2e4a45d14edfa755ce535be705705f813aa7c5ce3b3c7c"}
May 28 15:59:34.258846 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:34.258790    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8krdd" podStartSLOduration=3.290955916 podStartE2EDuration="22.258774835s" podCreationTimestamp="2026-05-28 15:59:12 +0000 UTC" firstStartedPulling="2026-05-28 15:59:13.29948731 +0000 UTC m=+1.859130785" lastFinishedPulling="2026-05-28 15:59:32.267306224 +0000 UTC m=+20.826949704" observedRunningTime="2026-05-28 15:59:34.258180389 +0000 UTC m=+22.817823887" watchObservedRunningTime="2026-05-28 15:59:34.258774835 +0000 UTC m=+22.818418333"
May 28 15:59:34.271282 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:34.271225    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal" podStartSLOduration=22.271205629 podStartE2EDuration="22.271205629s" podCreationTimestamp="2026-05-28 15:59:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 15:59:34.271117636 +0000 UTC m=+22.830761236" watchObservedRunningTime="2026-05-28 15:59:34.271205629 +0000 UTC m=+22.830849128"
May 28 15:59:34.942953 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:34.942923    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vj78z"
May 28 15:59:34.943473 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:34.943457    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vj78z"
May 28 15:59:35.070157 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:35.070118    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:35.070583 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:35.070118    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:35.070583 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:35.070233    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lw6n8" podUID="c73529ee-0f31-4285-85c2-51cdfd913e80"
May 28 15:59:35.070583 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:35.070253    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 15:59:35.070583 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:35.070405    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69vjf" podUID="bea98d0f-e54c-4da9-a207-f692c8781bea"
May 28 15:59:35.070583 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:35.070498    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bkvg9" podUID="369ece78-a49b-484b-8521-d1f4656e1c6e"
May 28 15:59:35.240498 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:35.240464    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8" event={"ID":"c4578fd3-b4bc-4b10-85df-d6f40b590795","Type":"ContainerStarted","Data":"fa202cce0075ee1179f23c98787f7a769219556eb4f99d4f903c2648cf7a516e"}
May 28 15:59:35.243740 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:35.243712    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp" event={"ID":"fd6dd018-b7d1-426f-ae3f-4d10ac882495","Type":"ContainerStarted","Data":"8f20b0648ce08909db935f6e9fd067c0eccd450ea969fb170796b3621de67225"}
May 28 15:59:36.004260 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:36.004224    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vj78z"
May 28 15:59:36.004939 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:36.004911    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vj78z"
May 28 15:59:36.017616 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:36.017565    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjwv8" podStartSLOduration=2.400447824 podStartE2EDuration="24.017548224s" podCreationTimestamp="2026-05-28 15:59:12 +0000 UTC" firstStartedPulling="2026-05-28 15:59:13.331992287 +0000 UTC m=+1.891635766" lastFinishedPulling="2026-05-28 15:59:34.949092689 +0000 UTC m=+23.508736166" observedRunningTime="2026-05-28 15:59:35.256254229 +0000 UTC m=+23.815897726" watchObservedRunningTime="2026-05-28 15:59:36.017548224 +0000 UTC m=+24.577191723"
May 28 15:59:36.944023 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:36.943988    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/369ece78-a49b-484b-8521-d1f4656e1c6e-original-pull-secret\") pod \"global-pull-secret-syncer-bkvg9\" (UID: \"369ece78-a49b-484b-8521-d1f4656e1c6e\") " pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 15:59:36.944710 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:36.944110    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 15:59:36.944710 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:36.944162    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/369ece78-a49b-484b-8521-d1f4656e1c6e-original-pull-secret podName:369ece78-a49b-484b-8521-d1f4656e1c6e nodeName:}" failed. No retries permitted until 2026-05-28 15:59:44.944149169 +0000 UTC m=+33.503792649 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/369ece78-a49b-484b-8521-d1f4656e1c6e-original-pull-secret") pod "global-pull-secret-syncer-bkvg9" (UID: "369ece78-a49b-484b-8521-d1f4656e1c6e") : object "kube-system"/"original-pull-secret" not registered
May 28 15:59:37.069783 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:37.069744    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:37.069951 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:37.069792    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 15:59:37.069951 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:37.069744    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:37.069951 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:37.069849    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lw6n8" podUID="c73529ee-0f31-4285-85c2-51cdfd913e80"
May 28 15:59:37.069951 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:37.069931    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69vjf" podUID="bea98d0f-e54c-4da9-a207-f692c8781bea"
May 28 15:59:37.070097 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:37.069991    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bkvg9" podUID="369ece78-a49b-484b-8521-d1f4656e1c6e"
May 28 15:59:38.257243 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:38.257201    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp" event={"ID":"fd6dd018-b7d1-426f-ae3f-4d10ac882495","Type":"ContainerStarted","Data":"99c66b03e4a168e572ed5c23dd48b077daa944dc209cc2f0cef3d61c9d14a6cf"}
May 28 15:59:38.257917 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:38.257545    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:38.274674 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:38.274646    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:38.282560 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:38.282511    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp" podStartSLOduration=7.329968577 podStartE2EDuration="26.282496503s" podCreationTimestamp="2026-05-28 15:59:12 +0000 UTC" firstStartedPulling="2026-05-28 15:59:13.38499823 +0000 UTC m=+1.944641706" lastFinishedPulling="2026-05-28 15:59:32.337526138 +0000 UTC m=+20.897169632" observedRunningTime="2026-05-28 15:59:38.281898644 +0000 UTC m=+26.841542142" watchObservedRunningTime="2026-05-28 15:59:38.282496503 +0000 UTC m=+26.842140004"
May 28 15:59:39.070000 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:39.069977    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:39.070110 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:39.069996    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:39.070110 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:39.069979    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 15:59:39.070200 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:39.070098    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lw6n8" podUID="c73529ee-0f31-4285-85c2-51cdfd913e80"
May 28 15:59:39.070200 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:39.070174    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bkvg9" podUID="369ece78-a49b-484b-8521-d1f4656e1c6e"
May 28 15:59:39.070288 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:39.070273    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69vjf" podUID="bea98d0f-e54c-4da9-a207-f692c8781bea"
May 28 15:59:39.181454 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:39.181429    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-69vjf"]
May 28 15:59:39.184001 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:39.183935    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bkvg9"]
May 28 15:59:39.184571 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:39.184551    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lw6n8"]
May 28 15:59:39.260946 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:39.260917    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fgmnc" event={"ID":"bdd26bcc-76e5-4f92-812a-d7d05a31e959","Type":"ContainerStarted","Data":"e77998d623208763216166b2054184f9c5c800c56f2fb19230741c59e4c77d80"}
May 28 15:59:39.260946 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:39.260941    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:39.261261 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:39.260969    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 15:59:39.261261 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:39.260989    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:39.261261 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:39.261074    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bkvg9" podUID="369ece78-a49b-484b-8521-d1f4656e1c6e"
May 28 15:59:39.261261 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:39.261134    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69vjf" podUID="bea98d0f-e54c-4da9-a207-f692c8781bea"
May 28 15:59:39.261261 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:39.261251    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lw6n8" podUID="c73529ee-0f31-4285-85c2-51cdfd913e80"
May 28 15:59:39.261462 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:39.261350    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:39.261462 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:39.261373    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:39.276483 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:39.276460    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 15:59:40.264624 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:40.264587    2559 generic.go:358] "Generic (PLEG): container finished" podID="bdd26bcc-76e5-4f92-812a-d7d05a31e959" containerID="e77998d623208763216166b2054184f9c5c800c56f2fb19230741c59e4c77d80" exitCode=0
May 28 15:59:40.265065 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:40.264641    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fgmnc" event={"ID":"bdd26bcc-76e5-4f92-812a-d7d05a31e959","Type":"ContainerDied","Data":"e77998d623208763216166b2054184f9c5c800c56f2fb19230741c59e4c77d80"}
May 28 15:59:41.069967 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:41.069932    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:41.069967 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:41.069966    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:41.070191 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:41.069970    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 15:59:41.070191 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:41.070061    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69vjf" podUID="bea98d0f-e54c-4da9-a207-f692c8781bea"
May 28 15:59:41.070191 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:41.070113    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lw6n8" podUID="c73529ee-0f31-4285-85c2-51cdfd913e80"
May 28 15:59:41.070191 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:41.070179    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bkvg9" podUID="369ece78-a49b-484b-8521-d1f4656e1c6e"
May 28 15:59:42.270768 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:42.270550    2559 generic.go:358] "Generic (PLEG): container finished" podID="bdd26bcc-76e5-4f92-812a-d7d05a31e959" containerID="e5b65f9b31a27e0647aff2d9b0ef3c081db8e126cf0ee104fb6ea1441f8b0bba" exitCode=0
May 28 15:59:42.271360 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:42.270631    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fgmnc" event={"ID":"bdd26bcc-76e5-4f92-812a-d7d05a31e959","Type":"ContainerDied","Data":"e5b65f9b31a27e0647aff2d9b0ef3c081db8e126cf0ee104fb6ea1441f8b0bba"}
May 28 15:59:43.069681 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:43.069647    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:43.069848 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:43.069647    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 15:59:43.069848 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:43.069752    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69vjf" podUID="bea98d0f-e54c-4da9-a207-f692c8781bea"
May 28 15:59:43.069848 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:43.069647    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:43.069848 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:43.069835    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bkvg9" podUID="369ece78-a49b-484b-8521-d1f4656e1c6e"
May 28 15:59:43.070013 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:43.069910    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lw6n8" podUID="c73529ee-0f31-4285-85c2-51cdfd913e80"
May 28 15:59:44.276019 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:44.275984    2559 generic.go:358] "Generic (PLEG): container finished" podID="bdd26bcc-76e5-4f92-812a-d7d05a31e959" containerID="18d8f801ccfe943ce91c2eb73d9be10241df494ff55931e84893b9202b45eef4" exitCode=0
May 28 15:59:44.276521 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:44.276048    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fgmnc" event={"ID":"bdd26bcc-76e5-4f92-812a-d7d05a31e959","Type":"ContainerDied","Data":"18d8f801ccfe943ce91c2eb73d9be10241df494ff55931e84893b9202b45eef4"}
May 28 15:59:44.702432 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:44.702340    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs\") pod \"network-metrics-daemon-69vjf\" (UID: \"bea98d0f-e54c-4da9-a207-f692c8781bea\") " pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:44.702432 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:44.702421    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-47886\" (UniqueName: \"kubernetes.io/projected/c73529ee-0f31-4285-85c2-51cdfd913e80-kube-api-access-47886\") pod \"network-check-target-lw6n8\" (UID: \"c73529ee-0f31-4285-85c2-51cdfd913e80\") " pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:44.702665 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:44.702533    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:59:44.702665 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:44.702614    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs podName:bea98d0f-e54c-4da9-a207-f692c8781bea nodeName:}" failed. No retries permitted until 2026-05-28 16:00:16.702591142 +0000 UTC m=+65.262234623 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs") pod "network-metrics-daemon-69vjf" (UID: "bea98d0f-e54c-4da9-a207-f692c8781bea") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 15:59:44.702665 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:44.702546    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 15:59:44.702665 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:44.702638    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 15:59:44.702665 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:44.702650    2559 projected.go:194] Error preparing data for projected volume kube-api-access-47886 for pod openshift-network-diagnostics/network-check-target-lw6n8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:59:44.702904 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:44.702701    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c73529ee-0f31-4285-85c2-51cdfd913e80-kube-api-access-47886 podName:c73529ee-0f31-4285-85c2-51cdfd913e80 nodeName:}" failed. No retries permitted until 2026-05-28 16:00:16.70268583 +0000 UTC m=+65.262329310 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-47886" (UniqueName: "kubernetes.io/projected/c73529ee-0f31-4285-85c2-51cdfd913e80-kube-api-access-47886") pod "network-check-target-lw6n8" (UID: "c73529ee-0f31-4285-85c2-51cdfd913e80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 15:59:45.005151 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:45.005119    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/369ece78-a49b-484b-8521-d1f4656e1c6e-original-pull-secret\") pod \"global-pull-secret-syncer-bkvg9\" (UID: \"369ece78-a49b-484b-8521-d1f4656e1c6e\") " pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 15:59:45.005357 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:45.005270    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 15:59:45.005357 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:45.005348    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/369ece78-a49b-484b-8521-d1f4656e1c6e-original-pull-secret podName:369ece78-a49b-484b-8521-d1f4656e1c6e nodeName:}" failed. No retries permitted until 2026-05-28 16:00:01.005314502 +0000 UTC m=+49.564957982 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/369ece78-a49b-484b-8521-d1f4656e1c6e-original-pull-secret") pod "global-pull-secret-syncer-bkvg9" (UID: "369ece78-a49b-484b-8521-d1f4656e1c6e") : object "kube-system"/"original-pull-secret" not registered
May 28 15:59:45.070011 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:45.069981    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:45.070195 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:45.069981    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 15:59:45.070195 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:45.070091    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lw6n8" podUID="c73529ee-0f31-4285-85c2-51cdfd913e80"
May 28 15:59:45.070195 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:45.069981    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:45.070195 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:45.070166    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bkvg9" podUID="369ece78-a49b-484b-8521-d1f4656e1c6e"
May 28 15:59:45.070417 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:45.070257    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69vjf" podUID="bea98d0f-e54c-4da9-a207-f692c8781bea"
May 28 15:59:46.204554 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.204483    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeReady"
May 28 15:59:46.204917 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.204602    2559 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 28 15:59:46.235264 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.235234    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d97985d-2269t"]
May 28 15:59:46.253532 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.253499    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c9c4c45fb-v4kbd"]
May 28 15:59:46.267542 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.267510    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-78499c94f5-9hjq9"]
May 28 15:59:46.267712 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.267644    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d97985d-2269t"
May 28 15:59:46.267775 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.267726    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c9c4c45fb-v4kbd"
May 28 15:59:46.270042 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.269831    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
May 28 15:59:46.270187 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.270112    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-5vj6l\""
May 28 15:59:46.270693 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.270344    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
May 28 15:59:46.270693 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.270373    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
May 28 15:59:46.270693 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.270401    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
May 28 15:59:46.270693 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.270535    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
May 28 15:59:46.281694 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.281668    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx"]
May 28 15:59:46.281891 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.281809    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:46.283874 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.283854    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 28 15:59:46.284169 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.284148    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 28 15:59:46.284265 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.284174    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dm5wd\""
May 28 15:59:46.284265 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.284219    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 28 15:59:46.291505 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.291339    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 28 15:59:46.295594 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.295571    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d97985d-2269t"]
May 28 15:59:46.295702 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.295603    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ddmdp"]
May 28 15:59:46.295764 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.295716    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx"
May 28 15:59:46.298013 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.297996    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
May 28 15:59:46.298202 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.298187    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
May 28 15:59:46.298280 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.298224    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
May 28 15:59:46.298280 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.298225    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
May 28 15:59:46.313801 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.313782    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c9c4c45fb-v4kbd"]
May 28 15:59:46.313801 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.313803    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx"]
May 28 15:59:46.313921 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.313812    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78499c94f5-9hjq9"]
May 28 15:59:46.313921 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.313819    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ddmdp"]
May 28 15:59:46.313921 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.313907    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ddmdp"
May 28 15:59:46.316049 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.316034    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 28 15:59:46.316129 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.316109    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 28 15:59:46.316192 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.316129    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9w77f\""
May 28 15:59:46.316192 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.316110    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 28 15:59:46.347907 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.347878    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-b2vxs"]
May 28 15:59:46.366611 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.366579    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b2vxs"]
May 28 15:59:46.366750 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.366635    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b2vxs"
May 28 15:59:46.368941 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.368917    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 28 15:59:46.369076 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.368946    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 28 15:59:46.369076 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.368967    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ptwfd\""
May 28 15:59:46.414784 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.414750    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g8mf\" (UniqueName: \"kubernetes.io/projected/fc16e686-037b-4387-a421-c5ef14bfc7f5-kube-api-access-9g8mf\") pod \"managed-serviceaccount-addon-agent-6d97985d-2269t\" (UID: \"fc16e686-037b-4387-a421-c5ef14bfc7f5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d97985d-2269t"
May 28 15:59:46.414939 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.414798    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e717510f-91c8-479f-84be-8ec7b1202691-ca\") pod \"cluster-proxy-proxy-agent-556fd7895f-xftpx\" (UID: \"e717510f-91c8-479f-84be-8ec7b1202691\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx"
May 28 15:59:46.414939 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.414852    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/796a53ef-2aff-4989-b714-f8643db1c5b8-image-registry-private-configuration\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:46.414939 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.414882    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9qj6\" (UniqueName: \"kubernetes.io/projected/e717510f-91c8-479f-84be-8ec7b1202691-kube-api-access-x9qj6\") pod \"cluster-proxy-proxy-agent-556fd7895f-xftpx\" (UID: \"e717510f-91c8-479f-84be-8ec7b1202691\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx"
May 28 15:59:46.414939 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.414904    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert\") pod \"ingress-canary-ddmdp\" (UID: \"6123b655-5980-4479-9c2a-829cde441719\") " pod="openshift-ingress-canary/ingress-canary-ddmdp"
May 28 15:59:46.414939 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.414937    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t7br\" (UniqueName: \"kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-kube-api-access-5t7br\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:46.415097 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.414972    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/796a53ef-2aff-4989-b714-f8643db1c5b8-ca-trust-extracted\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:46.415097 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.414998    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/796a53ef-2aff-4989-b714-f8643db1c5b8-trusted-ca\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:46.415097 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.415015    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/796a53ef-2aff-4989-b714-f8643db1c5b8-installation-pull-secrets\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:46.415097 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.415046    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e717510f-91c8-479f-84be-8ec7b1202691-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-556fd7895f-xftpx\" (UID: \"e717510f-91c8-479f-84be-8ec7b1202691\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx"
May 28 15:59:46.415097 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.415063    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fc16e686-037b-4387-a421-c5ef14bfc7f5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6d97985d-2269t\" (UID: \"fc16e686-037b-4387-a421-c5ef14bfc7f5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d97985d-2269t"
May 28 15:59:46.415097 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.415079    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbrsb\" (UniqueName: \"kubernetes.io/projected/33449810-e5cd-4fc7-b145-c3ebf5e1618b-kube-api-access-dbrsb\") pod \"klusterlet-addon-workmgr-7c9c4c45fb-v4kbd\" (UID: \"33449810-e5cd-4fc7-b145-c3ebf5e1618b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c9c4c45fb-v4kbd"
May 28 15:59:46.415097 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.415095    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e717510f-91c8-479f-84be-8ec7b1202691-hub\") pod \"cluster-proxy-proxy-agent-556fd7895f-xftpx\" (UID: \"e717510f-91c8-479f-84be-8ec7b1202691\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx"
May 28 15:59:46.415300 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.415110    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/33449810-e5cd-4fc7-b145-c3ebf5e1618b-klusterlet-config\") pod \"klusterlet-addon-workmgr-7c9c4c45fb-v4kbd\" (UID: \"33449810-e5cd-4fc7-b145-c3ebf5e1618b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c9c4c45fb-v4kbd"
May 28 15:59:46.415300 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.415141    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:46.415300 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.415170    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/e717510f-91c8-479f-84be-8ec7b1202691-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-556fd7895f-xftpx\" (UID: \"e717510f-91c8-479f-84be-8ec7b1202691\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx"
May 28 15:59:46.415300 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.415194    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e717510f-91c8-479f-84be-8ec7b1202691-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-556fd7895f-xftpx\" (UID: \"e717510f-91c8-479f-84be-8ec7b1202691\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx"
May 28 15:59:46.415300 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.415211    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm9bw\" (UniqueName: \"kubernetes.io/projected/6123b655-5980-4479-9c2a-829cde441719-kube-api-access-sm9bw\") pod \"ingress-canary-ddmdp\" (UID: \"6123b655-5980-4479-9c2a-829cde441719\") " pod="openshift-ingress-canary/ingress-canary-ddmdp"
May 28 15:59:46.415300 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.415226    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33449810-e5cd-4fc7-b145-c3ebf5e1618b-tmp\") pod \"klusterlet-addon-workmgr-7c9c4c45fb-v4kbd\" (UID: \"33449810-e5cd-4fc7-b145-c3ebf5e1618b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c9c4c45fb-v4kbd"
May 28 15:59:46.415300 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.415248    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-bound-sa-token\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:46.415300 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.415271    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-certificates\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:46.515915 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.515875    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-bound-sa-token\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:46.516105 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.515926    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-certificates\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:46.516105 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.515957    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-298bc\" (UniqueName: \"kubernetes.io/projected/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-kube-api-access-298bc\") pod \"dns-default-b2vxs\" (UID: \"a45a3b0c-89c4-4212-967e-a1b96d0c8a81\") " pod="openshift-dns/dns-default-b2vxs"
May 28 15:59:46.516105 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.516002    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9g8mf\" (UniqueName: \"kubernetes.io/projected/fc16e686-037b-4387-a421-c5ef14bfc7f5-kube-api-access-9g8mf\") pod \"managed-serviceaccount-addon-agent-6d97985d-2269t\" (UID: \"fc16e686-037b-4387-a421-c5ef14bfc7f5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d97985d-2269t"
May 28 15:59:46.516105 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.516034    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-tmp-dir\") pod \"dns-default-b2vxs\" (UID: \"a45a3b0c-89c4-4212-967e-a1b96d0c8a81\") " pod="openshift-dns/dns-default-b2vxs"
May 28 15:59:46.516105 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.516079    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e717510f-91c8-479f-84be-8ec7b1202691-ca\") pod \"cluster-proxy-proxy-agent-556fd7895f-xftpx\" (UID: \"e717510f-91c8-479f-84be-8ec7b1202691\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx"
May 28 15:59:46.516105 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.516099    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/796a53ef-2aff-4989-b714-f8643db1c5b8-image-registry-private-configuration\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:46.516740 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.516116    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x9qj6\" (UniqueName: \"kubernetes.io/projected/e717510f-91c8-479f-84be-8ec7b1202691-kube-api-access-x9qj6\") pod \"cluster-proxy-proxy-agent-556fd7895f-xftpx\" (UID: \"e717510f-91c8-479f-84be-8ec7b1202691\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx"
May 28 15:59:46.516740 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.516134    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert\") pod \"ingress-canary-ddmdp\" (UID: \"6123b655-5980-4479-9c2a-829cde441719\") " pod="openshift-ingress-canary/ingress-canary-ddmdp"
May 28 15:59:46.516740 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.516174    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5t7br\" (UniqueName: \"kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-kube-api-access-5t7br\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:46.516740 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.516194    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/796a53ef-2aff-4989-b714-f8643db1c5b8-ca-trust-extracted\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:46.516740 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.516225    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/796a53ef-2aff-4989-b714-f8643db1c5b8-trusted-ca\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:46.516740 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.516253    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/796a53ef-2aff-4989-b714-f8643db1c5b8-installation-pull-secrets\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:46.516740 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.516277    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e717510f-91c8-479f-84be-8ec7b1202691-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-556fd7895f-xftpx\" (UID: \"e717510f-91c8-479f-84be-8ec7b1202691\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx"
May 28 15:59:46.516740 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.516302    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fc16e686-037b-4387-a421-c5ef14bfc7f5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6d97985d-2269t\" (UID: \"fc16e686-037b-4387-a421-c5ef14bfc7f5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d97985d-2269t"
May 28 15:59:46.516740 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.516343    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dbrsb\" (UniqueName: \"kubernetes.io/projected/33449810-e5cd-4fc7-b145-c3ebf5e1618b-kube-api-access-dbrsb\") pod \"klusterlet-addon-workmgr-7c9c4c45fb-v4kbd\" (UID: \"33449810-e5cd-4fc7-b145-c3ebf5e1618b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c9c4c45fb-v4kbd"
May 28 15:59:46.516740 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.516412    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e717510f-91c8-479f-84be-8ec7b1202691-hub\") pod \"cluster-proxy-proxy-agent-556fd7895f-xftpx\" (UID: \"e717510f-91c8-479f-84be-8ec7b1202691\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx"
May 28 15:59:46.516740 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.516438    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/33449810-e5cd-4fc7-b145-c3ebf5e1618b-klusterlet-config\") pod \"klusterlet-addon-workmgr-7c9c4c45fb-v4kbd\" (UID: \"33449810-e5cd-4fc7-b145-c3ebf5e1618b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c9c4c45fb-v4kbd"
May 28 15:59:46.516740 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:46.516449    2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 15:59:46.516740 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.516467    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls\") pod \"dns-default-b2vxs\" (UID: \"a45a3b0c-89c4-4212-967e-a1b96d0c8a81\") " pod="openshift-dns/dns-default-b2vxs"
May 28 15:59:46.516740 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.516496    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:46.516740 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:46.516515    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert podName:6123b655-5980-4479-9c2a-829cde441719 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:47.016495701 +0000 UTC m=+35.576139191 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert") pod "ingress-canary-ddmdp" (UID: "6123b655-5980-4479-9c2a-829cde441719") : secret "canary-serving-cert" not found
May 28 15:59:46.516740 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.516552    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/e717510f-91c8-479f-84be-8ec7b1202691-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-556fd7895f-xftpx\" (UID: \"e717510f-91c8-479f-84be-8ec7b1202691\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx"
May 28 15:59:46.517556 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:46.516567    2559 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 15:59:46.517556 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:46.516580    2559 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78499c94f5-9hjq9: secret "image-registry-tls" not found
May 28 15:59:46.517556 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.516583    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e717510f-91c8-479f-84be-8ec7b1202691-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-556fd7895f-xftpx\" (UID: \"e717510f-91c8-479f-84be-8ec7b1202691\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx"
May 28 15:59:46.517556 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.516611    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sm9bw\" (UniqueName: \"kubernetes.io/projected/6123b655-5980-4479-9c2a-829cde441719-kube-api-access-sm9bw\") pod \"ingress-canary-ddmdp\" (UID: \"6123b655-5980-4479-9c2a-829cde441719\") " pod="openshift-ingress-canary/ingress-canary-ddmdp"
May 28 15:59:46.517556 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:46.516620    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls podName:796a53ef-2aff-4989-b714-f8643db1c5b8 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:47.01660606 +0000 UTC m=+35.576249560 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls") pod "image-registry-78499c94f5-9hjq9" (UID: "796a53ef-2aff-4989-b714-f8643db1c5b8") : secret "image-registry-tls" not found
May 28 15:59:46.517556 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.516632    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/796a53ef-2aff-4989-b714-f8643db1c5b8-ca-trust-extracted\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:46.517556 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.516652    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33449810-e5cd-4fc7-b145-c3ebf5e1618b-tmp\") pod \"klusterlet-addon-workmgr-7c9c4c45fb-v4kbd\" (UID: \"33449810-e5cd-4fc7-b145-c3ebf5e1618b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c9c4c45fb-v4kbd"
May 28 15:59:46.517556 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.516679    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-config-volume\") pod \"dns-default-b2vxs\" (UID: \"a45a3b0c-89c4-4212-967e-a1b96d0c8a81\") " pod="openshift-dns/dns-default-b2vxs"
May 28 15:59:46.517556 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.516712    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-certificates\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:46.517983 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.517584    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/796a53ef-2aff-4989-b714-f8643db1c5b8-trusted-ca\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:46.517983 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.517674    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/e717510f-91c8-479f-84be-8ec7b1202691-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-556fd7895f-xftpx\" (UID: \"e717510f-91c8-479f-84be-8ec7b1202691\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx"
May 28 15:59:46.518779 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.518440    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33449810-e5cd-4fc7-b145-c3ebf5e1618b-tmp\") pod \"klusterlet-addon-workmgr-7c9c4c45fb-v4kbd\" (UID: \"33449810-e5cd-4fc7-b145-c3ebf5e1618b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c9c4c45fb-v4kbd"
May 28 15:59:46.522013 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.521981    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/796a53ef-2aff-4989-b714-f8643db1c5b8-installation-pull-secrets\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:46.522127 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.522034    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/33449810-e5cd-4fc7-b145-c3ebf5e1618b-klusterlet-config\") pod \"klusterlet-addon-workmgr-7c9c4c45fb-v4kbd\" (UID: \"33449810-e5cd-4fc7-b145-c3ebf5e1618b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c9c4c45fb-v4kbd"
May 28 15:59:46.522127 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.522054    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e717510f-91c8-479f-84be-8ec7b1202691-ca\") pod \"cluster-proxy-proxy-agent-556fd7895f-xftpx\" (UID: \"e717510f-91c8-479f-84be-8ec7b1202691\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx"
May 28 15:59:46.522127 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.522107    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e717510f-91c8-479f-84be-8ec7b1202691-hub\") pod \"cluster-proxy-proxy-agent-556fd7895f-xftpx\" (UID: \"e717510f-91c8-479f-84be-8ec7b1202691\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx"
May 28 15:59:46.522292 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.522202    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e717510f-91c8-479f-84be-8ec7b1202691-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-556fd7895f-xftpx\" (UID: \"e717510f-91c8-479f-84be-8ec7b1202691\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx"
May 28 15:59:46.522292 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.522210    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fc16e686-037b-4387-a421-c5ef14bfc7f5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6d97985d-2269t\" (UID: \"fc16e686-037b-4387-a421-c5ef14bfc7f5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d97985d-2269t"
May 28 15:59:46.522408 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.522292    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/796a53ef-2aff-4989-b714-f8643db1c5b8-image-registry-private-configuration\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:46.524014 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.523952    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e717510f-91c8-479f-84be-8ec7b1202691-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-556fd7895f-xftpx\" (UID: \"e717510f-91c8-479f-84be-8ec7b1202691\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx"
May 28 15:59:46.525383 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.525356    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t7br\" (UniqueName: \"kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-kube-api-access-5t7br\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:46.526071 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.525895    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9qj6\" (UniqueName: \"kubernetes.io/projected/e717510f-91c8-479f-84be-8ec7b1202691-kube-api-access-x9qj6\") pod \"cluster-proxy-proxy-agent-556fd7895f-xftpx\" (UID: \"e717510f-91c8-479f-84be-8ec7b1202691\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx"
May 28 15:59:46.526071 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.526014    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g8mf\" (UniqueName: \"kubernetes.io/projected/fc16e686-037b-4387-a421-c5ef14bfc7f5-kube-api-access-9g8mf\") pod \"managed-serviceaccount-addon-agent-6d97985d-2269t\" (UID: \"fc16e686-037b-4387-a421-c5ef14bfc7f5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d97985d-2269t"
May 28 15:59:46.526206 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.526101    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-bound-sa-token\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:46.526206 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.526104    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm9bw\" (UniqueName: \"kubernetes.io/projected/6123b655-5980-4479-9c2a-829cde441719-kube-api-access-sm9bw\") pod \"ingress-canary-ddmdp\" (UID: \"6123b655-5980-4479-9c2a-829cde441719\") " pod="openshift-ingress-canary/ingress-canary-ddmdp"
May 28 15:59:46.526387 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.526364    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbrsb\" (UniqueName: \"kubernetes.io/projected/33449810-e5cd-4fc7-b145-c3ebf5e1618b-kube-api-access-dbrsb\") pod \"klusterlet-addon-workmgr-7c9c4c45fb-v4kbd\" (UID: \"33449810-e5cd-4fc7-b145-c3ebf5e1618b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c9c4c45fb-v4kbd"
May 28 15:59:46.586279 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.586247    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d97985d-2269t"
May 28 15:59:46.595135 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.595107    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c9c4c45fb-v4kbd"
May 28 15:59:46.607648 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.607599    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx"
May 28 15:59:46.617932 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.617900    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls\") pod \"dns-default-b2vxs\" (UID: \"a45a3b0c-89c4-4212-967e-a1b96d0c8a81\") " pod="openshift-dns/dns-default-b2vxs"
May 28 15:59:46.618065 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.617975    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-config-volume\") pod \"dns-default-b2vxs\" (UID: \"a45a3b0c-89c4-4212-967e-a1b96d0c8a81\") " pod="openshift-dns/dns-default-b2vxs"
May 28 15:59:46.618065 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.618009    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-298bc\" (UniqueName: \"kubernetes.io/projected/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-kube-api-access-298bc\") pod \"dns-default-b2vxs\" (UID: \"a45a3b0c-89c4-4212-967e-a1b96d0c8a81\") " pod="openshift-dns/dns-default-b2vxs"
May 28 15:59:46.618163 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.618073    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-tmp-dir\") pod \"dns-default-b2vxs\" (UID: \"a45a3b0c-89c4-4212-967e-a1b96d0c8a81\") " pod="openshift-dns/dns-default-b2vxs"
May 28 15:59:46.618609 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:46.618415    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 15:59:46.618609 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:46.618497    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls podName:a45a3b0c-89c4-4212-967e-a1b96d0c8a81 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:47.118477408 +0000 UTC m=+35.678120918 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls") pod "dns-default-b2vxs" (UID: "a45a3b0c-89c4-4212-967e-a1b96d0c8a81") : secret "dns-default-metrics-tls" not found
May 28 15:59:46.618609 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.618486    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-tmp-dir\") pod \"dns-default-b2vxs\" (UID: \"a45a3b0c-89c4-4212-967e-a1b96d0c8a81\") " pod="openshift-dns/dns-default-b2vxs"
May 28 15:59:46.618830 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.618616    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-config-volume\") pod \"dns-default-b2vxs\" (UID: \"a45a3b0c-89c4-4212-967e-a1b96d0c8a81\") " pod="openshift-dns/dns-default-b2vxs"
May 28 15:59:46.627097 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.627041    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-298bc\" (UniqueName: \"kubernetes.io/projected/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-kube-api-access-298bc\") pod \"dns-default-b2vxs\" (UID: \"a45a3b0c-89c4-4212-967e-a1b96d0c8a81\") " pod="openshift-dns/dns-default-b2vxs"
May 28 15:59:46.808345 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.807767    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d97985d-2269t"]
May 28 15:59:46.809167 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.808803    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c9c4c45fb-v4kbd"]
May 28 15:59:46.810696 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:46.810660    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx"]
May 28 15:59:46.812660 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:46.812628    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc16e686_037b_4387_a421_c5ef14bfc7f5.slice/crio-6b21103480ea8c0dbca7f1235ac58a992d7c36d1cd9e8747fdc804b30292192f WatchSource:0}: Error finding container 6b21103480ea8c0dbca7f1235ac58a992d7c36d1cd9e8747fdc804b30292192f: Status 404 returned error can't find the container with id 6b21103480ea8c0dbca7f1235ac58a992d7c36d1cd9e8747fdc804b30292192f
May 28 15:59:46.813050 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:46.813000    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33449810_e5cd_4fc7_b145_c3ebf5e1618b.slice/crio-c300079571c862a85e527f1233521e51925045bfd706ad1540af3c9004883f6e WatchSource:0}: Error finding container c300079571c862a85e527f1233521e51925045bfd706ad1540af3c9004883f6e: Status 404 returned error can't find the container with id c300079571c862a85e527f1233521e51925045bfd706ad1540af3c9004883f6e
May 28 15:59:46.814013 ip-10-0-133-120 kubenswrapper[2559]: W0528 15:59:46.813992    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode717510f_91c8_479f_84be_8ec7b1202691.slice/crio-018b797cd44b574b4b2423689b994f0426a652e6e3c3aac93113290840369c56 WatchSource:0}: Error finding container 018b797cd44b574b4b2423689b994f0426a652e6e3c3aac93113290840369c56: Status 404 returned error can't find the container with id 018b797cd44b574b4b2423689b994f0426a652e6e3c3aac93113290840369c56
May 28 15:59:47.020973 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:47.020938    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert\") pod \"ingress-canary-ddmdp\" (UID: \"6123b655-5980-4479-9c2a-829cde441719\") " pod="openshift-ingress-canary/ingress-canary-ddmdp"
May 28 15:59:47.021149 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:47.021021    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:47.021149 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:47.021093    2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 15:59:47.021149 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:47.021123    2559 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 15:59:47.021149 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:47.021138    2559 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78499c94f5-9hjq9: secret "image-registry-tls" not found
May 28 15:59:47.021350 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:47.021153    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert podName:6123b655-5980-4479-9c2a-829cde441719 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:48.021136091 +0000 UTC m=+36.580779586 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert") pod "ingress-canary-ddmdp" (UID: "6123b655-5980-4479-9c2a-829cde441719") : secret "canary-serving-cert" not found
May 28 15:59:47.021350 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:47.021183    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls podName:796a53ef-2aff-4989-b714-f8643db1c5b8 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:48.021167849 +0000 UTC m=+36.580811331 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls") pod "image-registry-78499c94f5-9hjq9" (UID: "796a53ef-2aff-4989-b714-f8643db1c5b8") : secret "image-registry-tls" not found
May 28 15:59:47.069811 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:47.069729    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 15:59:47.069970 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:47.069735    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 15:59:47.070152 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:47.070137    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 15:59:47.072459 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:47.072424    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 28 15:59:47.072551 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:47.072474    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 15:59:47.073034 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:47.072881    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4ck6s\""
May 28 15:59:47.073034 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:47.072888    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 15:59:47.073034 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:47.072991    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 15:59:47.073257 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:47.073101    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6wrpt\""
May 28 15:59:47.121458 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:47.121424    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls\") pod \"dns-default-b2vxs\" (UID: \"a45a3b0c-89c4-4212-967e-a1b96d0c8a81\") " pod="openshift-dns/dns-default-b2vxs"
May 28 15:59:47.121590 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:47.121562    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 15:59:47.121641 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:47.121628    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls podName:a45a3b0c-89c4-4212-967e-a1b96d0c8a81 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:48.121611231 +0000 UTC m=+36.681254707 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls") pod "dns-default-b2vxs" (UID: "a45a3b0c-89c4-4212-967e-a1b96d0c8a81") : secret "dns-default-metrics-tls" not found
May 28 15:59:47.281930 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:47.281896    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c9c4c45fb-v4kbd" event={"ID":"33449810-e5cd-4fc7-b145-c3ebf5e1618b","Type":"ContainerStarted","Data":"c300079571c862a85e527f1233521e51925045bfd706ad1540af3c9004883f6e"}
May 28 15:59:47.282828 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:47.282805    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx" event={"ID":"e717510f-91c8-479f-84be-8ec7b1202691","Type":"ContainerStarted","Data":"018b797cd44b574b4b2423689b994f0426a652e6e3c3aac93113290840369c56"}
May 28 15:59:47.283673 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:47.283655    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d97985d-2269t" event={"ID":"fc16e686-037b-4387-a421-c5ef14bfc7f5","Type":"ContainerStarted","Data":"6b21103480ea8c0dbca7f1235ac58a992d7c36d1cd9e8747fdc804b30292192f"}
May 28 15:59:48.028511 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:48.027615    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:48.028511 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:48.027728    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert\") pod \"ingress-canary-ddmdp\" (UID: \"6123b655-5980-4479-9c2a-829cde441719\") " pod="openshift-ingress-canary/ingress-canary-ddmdp"
May 28 15:59:48.028511 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:48.027996    2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 15:59:48.028511 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:48.028067    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert podName:6123b655-5980-4479-9c2a-829cde441719 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:50.028047363 +0000 UTC m=+38.587690845 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert") pod "ingress-canary-ddmdp" (UID: "6123b655-5980-4479-9c2a-829cde441719") : secret "canary-serving-cert" not found
May 28 15:59:48.029131 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:48.029003    2559 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 15:59:48.029131 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:48.029025    2559 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78499c94f5-9hjq9: secret "image-registry-tls" not found
May 28 15:59:48.029131 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:48.029076    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls podName:796a53ef-2aff-4989-b714-f8643db1c5b8 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:50.029061105 +0000 UTC m=+38.588704586 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls") pod "image-registry-78499c94f5-9hjq9" (UID: "796a53ef-2aff-4989-b714-f8643db1c5b8") : secret "image-registry-tls" not found
May 28 15:59:48.129081 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:48.129041    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls\") pod \"dns-default-b2vxs\" (UID: \"a45a3b0c-89c4-4212-967e-a1b96d0c8a81\") " pod="openshift-dns/dns-default-b2vxs"
May 28 15:59:48.129305 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:48.129227    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 15:59:48.129305 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:48.129294    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls podName:a45a3b0c-89c4-4212-967e-a1b96d0c8a81 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:50.129274787 +0000 UTC m=+38.688918275 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls") pod "dns-default-b2vxs" (UID: "a45a3b0c-89c4-4212-967e-a1b96d0c8a81") : secret "dns-default-metrics-tls" not found
May 28 15:59:50.048272 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:50.048233    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert\") pod \"ingress-canary-ddmdp\" (UID: \"6123b655-5980-4479-9c2a-829cde441719\") " pod="openshift-ingress-canary/ingress-canary-ddmdp"
May 28 15:59:50.048707 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:50.048357    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:50.048707 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:50.048510    2559 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 15:59:50.048707 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:50.048522    2559 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78499c94f5-9hjq9: secret "image-registry-tls" not found
May 28 15:59:50.048707 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:50.048575    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls podName:796a53ef-2aff-4989-b714-f8643db1c5b8 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:54.048556604 +0000 UTC m=+42.608200090 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls") pod "image-registry-78499c94f5-9hjq9" (UID: "796a53ef-2aff-4989-b714-f8643db1c5b8") : secret "image-registry-tls" not found
May 28 15:59:50.048992 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:50.048963    2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 15:59:50.049054 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:50.049024    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert podName:6123b655-5980-4479-9c2a-829cde441719 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:54.049008265 +0000 UTC m=+42.608651744 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert") pod "ingress-canary-ddmdp" (UID: "6123b655-5980-4479-9c2a-829cde441719") : secret "canary-serving-cert" not found
May 28 15:59:50.149571 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:50.149538    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls\") pod \"dns-default-b2vxs\" (UID: \"a45a3b0c-89c4-4212-967e-a1b96d0c8a81\") " pod="openshift-dns/dns-default-b2vxs"
May 28 15:59:50.149739 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:50.149694    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 15:59:50.149779 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:50.149753    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls podName:a45a3b0c-89c4-4212-967e-a1b96d0c8a81 nodeName:}" failed. No retries permitted until 2026-05-28 15:59:54.14973843 +0000 UTC m=+42.709381912 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls") pod "dns-default-b2vxs" (UID: "a45a3b0c-89c4-4212-967e-a1b96d0c8a81") : secret "dns-default-metrics-tls" not found
May 28 15:59:54.082255 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:54.082219    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert\") pod \"ingress-canary-ddmdp\" (UID: \"6123b655-5980-4479-9c2a-829cde441719\") " pod="openshift-ingress-canary/ingress-canary-ddmdp"
May 28 15:59:54.082724 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:54.082340    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 15:59:54.082724 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:54.082391    2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 15:59:54.082724 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:54.082459    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert podName:6123b655-5980-4479-9c2a-829cde441719 nodeName:}" failed. No retries permitted until 2026-05-28 16:00:02.082441415 +0000 UTC m=+50.642084892 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert") pod "ingress-canary-ddmdp" (UID: "6123b655-5980-4479-9c2a-829cde441719") : secret "canary-serving-cert" not found
May 28 15:59:54.082724 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:54.082489    2559 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 15:59:54.082724 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:54.082504    2559 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78499c94f5-9hjq9: secret "image-registry-tls" not found
May 28 15:59:54.082724 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:54.082562    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls podName:796a53ef-2aff-4989-b714-f8643db1c5b8 nodeName:}" failed. No retries permitted until 2026-05-28 16:00:02.082543506 +0000 UTC m=+50.642186984 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls") pod "image-registry-78499c94f5-9hjq9" (UID: "796a53ef-2aff-4989-b714-f8643db1c5b8") : secret "image-registry-tls" not found
May 28 15:59:54.183158 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:54.183120    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls\") pod \"dns-default-b2vxs\" (UID: \"a45a3b0c-89c4-4212-967e-a1b96d0c8a81\") " pod="openshift-dns/dns-default-b2vxs"
May 28 15:59:54.183295 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:54.183276    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 15:59:54.183379 ip-10-0-133-120 kubenswrapper[2559]: E0528 15:59:54.183368    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls podName:a45a3b0c-89c4-4212-967e-a1b96d0c8a81 nodeName:}" failed. No retries permitted until 2026-05-28 16:00:02.183350869 +0000 UTC m=+50.742994345 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls") pod "dns-default-b2vxs" (UID: "a45a3b0c-89c4-4212-967e-a1b96d0c8a81") : secret "dns-default-metrics-tls" not found
May 28 15:59:58.316381 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:58.316306    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c9c4c45fb-v4kbd" event={"ID":"33449810-e5cd-4fc7-b145-c3ebf5e1618b","Type":"ContainerStarted","Data":"fa1c840c7e02375bb747260753f2bd713d6eccec080444b429ecca9ecd016a44"}
May 28 15:59:58.316854 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:58.316516    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c9c4c45fb-v4kbd"
May 28 15:59:58.318378 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:58.318355    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c9c4c45fb-v4kbd"
May 28 15:59:58.319097 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:58.319073    2559 generic.go:358] "Generic (PLEG): container finished" podID="bdd26bcc-76e5-4f92-812a-d7d05a31e959" containerID="55483dfdaa9314404a56647578390fa645f08716e8d9eae235f87cb83ba8243e" exitCode=0
May 28 15:59:58.319169 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:58.319144    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fgmnc" event={"ID":"bdd26bcc-76e5-4f92-812a-d7d05a31e959","Type":"ContainerDied","Data":"55483dfdaa9314404a56647578390fa645f08716e8d9eae235f87cb83ba8243e"}
May 28 15:59:58.320404 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:58.320384    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx" event={"ID":"e717510f-91c8-479f-84be-8ec7b1202691","Type":"ContainerStarted","Data":"e799df51b37e7d045dfde04afc7b01fa38566771e5271f6c67679a5b5452db6f"}
May 28 15:59:58.321583 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:58.321563    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d97985d-2269t" event={"ID":"fc16e686-037b-4387-a421-c5ef14bfc7f5","Type":"ContainerStarted","Data":"ac3b1e624e89f6861a47360d56adf721c297b96fdb8ebe4382a2139f12869ce0"}
May 28 15:59:58.331138 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:58.331099    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c9c4c45fb-v4kbd" podStartSLOduration=16.702807347 podStartE2EDuration="27.331086078s" podCreationTimestamp="2026-05-28 15:59:31 +0000 UTC" firstStartedPulling="2026-05-28 15:59:46.815021735 +0000 UTC m=+35.374665213" lastFinishedPulling="2026-05-28 15:59:57.443300463 +0000 UTC m=+46.002943944" observedRunningTime="2026-05-28 15:59:58.330065233 +0000 UTC m=+46.889708730" watchObservedRunningTime="2026-05-28 15:59:58.331086078 +0000 UTC m=+46.890729576"
May 28 15:59:58.365366 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:58.365024    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d97985d-2269t" podStartSLOduration=16.729607835 podStartE2EDuration="27.365003268s" podCreationTimestamp="2026-05-28 15:59:31 +0000 UTC" firstStartedPulling="2026-05-28 15:59:46.814808432 +0000 UTC m=+35.374451909" lastFinishedPulling="2026-05-28 15:59:57.450203866 +0000 UTC m=+46.009847342" observedRunningTime="2026-05-28 15:59:58.364214704 +0000 UTC m=+46.923858205" watchObservedRunningTime="2026-05-28 15:59:58.365003268 +0000 UTC m=+46.924646767"
May 28 15:59:59.327389 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:59.327351    2559 generic.go:358] "Generic (PLEG): container finished" podID="bdd26bcc-76e5-4f92-812a-d7d05a31e959" containerID="659b727c0f7702497b62c35f55f98e6d8cf13642f2d9c3cb0aabc596d0263286" exitCode=0
May 28 15:59:59.327855 ip-10-0-133-120 kubenswrapper[2559]: I0528 15:59:59.327443    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fgmnc" event={"ID":"bdd26bcc-76e5-4f92-812a-d7d05a31e959","Type":"ContainerDied","Data":"659b727c0f7702497b62c35f55f98e6d8cf13642f2d9c3cb0aabc596d0263286"}
May 28 16:00:00.330762 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:00.330649    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx" event={"ID":"e717510f-91c8-479f-84be-8ec7b1202691","Type":"ContainerStarted","Data":"29fc3a7d61cbdf57512bb5faa32887ba4829a7a8e5eddedb4fede0169e1c9fac"}
May 28 16:00:00.330762 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:00.330699    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx" event={"ID":"e717510f-91c8-479f-84be-8ec7b1202691","Type":"ContainerStarted","Data":"e8e67f58fbe07e82f596730797b06f7225128bd94b5a9a49058e3dc7ab3c8f57"}
May 28 16:00:00.333347 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:00.333262    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fgmnc" event={"ID":"bdd26bcc-76e5-4f92-812a-d7d05a31e959","Type":"ContainerStarted","Data":"2f53155e85057fd4ed15f8990fd3ba1f2f1b0cb02018c27f7a81caf89382d988"}
May 28 16:00:00.350128 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:00.350084    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx" podStartSLOduration=16.142924517 podStartE2EDuration="29.350072162s" podCreationTimestamp="2026-05-28 15:59:31 +0000 UTC" firstStartedPulling="2026-05-28 15:59:46.815970471 +0000 UTC m=+35.375613961" lastFinishedPulling="2026-05-28 16:00:00.023118128 +0000 UTC m=+48.582761606" observedRunningTime="2026-05-28 16:00:00.348284268 +0000 UTC m=+48.907927765" watchObservedRunningTime="2026-05-28 16:00:00.350072162 +0000 UTC m=+48.909715701"
May 28 16:00:00.368314 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:00.368270    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fgmnc" podStartSLOduration=4.294803553 podStartE2EDuration="48.368257605s" podCreationTimestamp="2026-05-28 15:59:12 +0000 UTC" firstStartedPulling="2026-05-28 15:59:13.370299364 +0000 UTC m=+1.929942840" lastFinishedPulling="2026-05-28 15:59:57.443753409 +0000 UTC m=+46.003396892" observedRunningTime="2026-05-28 16:00:00.367292056 +0000 UTC m=+48.926935556" watchObservedRunningTime="2026-05-28 16:00:00.368257605 +0000 UTC m=+48.927901102"
May 28 16:00:01.051579 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:01.051532    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/369ece78-a49b-484b-8521-d1f4656e1c6e-original-pull-secret\") pod \"global-pull-secret-syncer-bkvg9\" (UID: \"369ece78-a49b-484b-8521-d1f4656e1c6e\") " pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 16:00:01.054669 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:01.054639    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/369ece78-a49b-484b-8521-d1f4656e1c6e-original-pull-secret\") pod \"global-pull-secret-syncer-bkvg9\" (UID: \"369ece78-a49b-484b-8521-d1f4656e1c6e\") " pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 16:00:01.182787 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:01.182748    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bkvg9"
May 28 16:00:01.299776 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:01.299741    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bkvg9"]
May 28 16:00:01.303190 ip-10-0-133-120 kubenswrapper[2559]: W0528 16:00:01.303133    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod369ece78_a49b_484b_8521_d1f4656e1c6e.slice/crio-a0a537bef94b46037c18d389d582150e08c6212fe391a11992b5651ce6c2519a WatchSource:0}: Error finding container a0a537bef94b46037c18d389d582150e08c6212fe391a11992b5651ce6c2519a: Status 404 returned error can't find the container with id a0a537bef94b46037c18d389d582150e08c6212fe391a11992b5651ce6c2519a
May 28 16:00:01.336478 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:01.336443    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bkvg9" event={"ID":"369ece78-a49b-484b-8521-d1f4656e1c6e","Type":"ContainerStarted","Data":"a0a537bef94b46037c18d389d582150e08c6212fe391a11992b5651ce6c2519a"}
May 28 16:00:02.160734 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:02.160676    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 16:00:02.160922 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:02.160782    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert\") pod \"ingress-canary-ddmdp\" (UID: \"6123b655-5980-4479-9c2a-829cde441719\") " pod="openshift-ingress-canary/ingress-canary-ddmdp"
May 28 16:00:02.160922 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:00:02.160843    2559 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 16:00:02.160922 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:00:02.160861    2559 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78499c94f5-9hjq9: secret "image-registry-tls" not found
May 28 16:00:02.160922 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:00:02.160879    2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 16:00:02.161055 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:00:02.160927    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert podName:6123b655-5980-4479-9c2a-829cde441719 nodeName:}" failed. No retries permitted until 2026-05-28 16:00:18.160906392 +0000 UTC m=+66.720549870 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert") pod "ingress-canary-ddmdp" (UID: "6123b655-5980-4479-9c2a-829cde441719") : secret "canary-serving-cert" not found
May 28 16:00:02.161055 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:00:02.160943    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls podName:796a53ef-2aff-4989-b714-f8643db1c5b8 nodeName:}" failed. No retries permitted until 2026-05-28 16:00:18.160935075 +0000 UTC m=+66.720578556 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls") pod "image-registry-78499c94f5-9hjq9" (UID: "796a53ef-2aff-4989-b714-f8643db1c5b8") : secret "image-registry-tls" not found
May 28 16:00:02.261793 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:02.261754    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls\") pod \"dns-default-b2vxs\" (UID: \"a45a3b0c-89c4-4212-967e-a1b96d0c8a81\") " pod="openshift-dns/dns-default-b2vxs"
May 28 16:00:02.261961 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:00:02.261900    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 16:00:02.262009 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:00:02.261962    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls podName:a45a3b0c-89c4-4212-967e-a1b96d0c8a81 nodeName:}" failed. No retries permitted until 2026-05-28 16:00:18.261944807 +0000 UTC m=+66.821588283 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls") pod "dns-default-b2vxs" (UID: "a45a3b0c-89c4-4212-967e-a1b96d0c8a81") : secret "dns-default-metrics-tls" not found
May 28 16:00:07.350984 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:07.350942    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bkvg9" event={"ID":"369ece78-a49b-484b-8521-d1f4656e1c6e","Type":"ContainerStarted","Data":"3e9f17a4dd721b96d3cd90e9ea5f14f131f5db8fb1dabf287891a1fe0641fc03"}
May 28 16:00:07.365182 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:07.365131    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-bkvg9" podStartSLOduration=33.026381196 podStartE2EDuration="38.365114048s" podCreationTimestamp="2026-05-28 15:59:29 +0000 UTC" firstStartedPulling="2026-05-28 16:00:01.30488127 +0000 UTC m=+49.864524759" lastFinishedPulling="2026-05-28 16:00:06.643614136 +0000 UTC m=+55.203257611" observedRunningTime="2026-05-28 16:00:07.364859142 +0000 UTC m=+55.924502640" watchObservedRunningTime="2026-05-28 16:00:07.365114048 +0000 UTC m=+55.924757547"
May 28 16:00:11.282101 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:11.282072    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bgmxp"
May 28 16:00:16.769107 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:16.769064    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-47886\" (UniqueName: \"kubernetes.io/projected/c73529ee-0f31-4285-85c2-51cdfd913e80-kube-api-access-47886\") pod \"network-check-target-lw6n8\" (UID: \"c73529ee-0f31-4285-85c2-51cdfd913e80\") " pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 16:00:16.769619 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:16.769147    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs\") pod \"network-metrics-daemon-69vjf\" (UID: \"bea98d0f-e54c-4da9-a207-f692c8781bea\") " pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 16:00:16.771827 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:16.771810    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 16:00:16.771872 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:16.771844    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 16:00:16.779580 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:00:16.779562    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 28 16:00:16.779668 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:00:16.779635    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs podName:bea98d0f-e54c-4da9-a207-f692c8781bea nodeName:}" failed. No retries permitted until 2026-05-28 16:01:20.779613496 +0000 UTC m=+129.339256987 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs") pod "network-metrics-daemon-69vjf" (UID: "bea98d0f-e54c-4da9-a207-f692c8781bea") : secret "metrics-daemon-secret" not found
May 28 16:00:16.781446 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:16.781427    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 16:00:16.791916 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:16.791896    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47886\" (UniqueName: \"kubernetes.io/projected/c73529ee-0f31-4285-85c2-51cdfd913e80-kube-api-access-47886\") pod \"network-check-target-lw6n8\" (UID: \"c73529ee-0f31-4285-85c2-51cdfd913e80\") " pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 16:00:16.799032 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:16.799014    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6wrpt\""
May 28 16:00:16.807804 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:16.807787    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 16:00:16.921594 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:16.921564    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lw6n8"]
May 28 16:00:16.924693 ip-10-0-133-120 kubenswrapper[2559]: W0528 16:00:16.924660    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc73529ee_0f31_4285_85c2_51cdfd913e80.slice/crio-20de91d6973ee1fdaddc14bb414e82bfdd7486946cf3da737a55b75813878e1f WatchSource:0}: Error finding container 20de91d6973ee1fdaddc14bb414e82bfdd7486946cf3da737a55b75813878e1f: Status 404 returned error can't find the container with id 20de91d6973ee1fdaddc14bb414e82bfdd7486946cf3da737a55b75813878e1f
May 28 16:00:17.375508 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:17.375465    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lw6n8" event={"ID":"c73529ee-0f31-4285-85c2-51cdfd913e80","Type":"ContainerStarted","Data":"20de91d6973ee1fdaddc14bb414e82bfdd7486946cf3da737a55b75813878e1f"}
May 28 16:00:18.180372 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:18.180307    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert\") pod \"ingress-canary-ddmdp\" (UID: \"6123b655-5980-4479-9c2a-829cde441719\") " pod="openshift-ingress-canary/ingress-canary-ddmdp"
May 28 16:00:18.180859 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:18.180429    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 16:00:18.180859 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:00:18.180471    2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 16:00:18.180859 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:00:18.180545    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert podName:6123b655-5980-4479-9c2a-829cde441719 nodeName:}" failed. No retries permitted until 2026-05-28 16:00:50.18052375 +0000 UTC m=+98.740167243 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert") pod "ingress-canary-ddmdp" (UID: "6123b655-5980-4479-9c2a-829cde441719") : secret "canary-serving-cert" not found
May 28 16:00:18.180859 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:00:18.180564    2559 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 16:00:18.180859 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:00:18.180578    2559 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78499c94f5-9hjq9: secret "image-registry-tls" not found
May 28 16:00:18.180859 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:00:18.180618    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls podName:796a53ef-2aff-4989-b714-f8643db1c5b8 nodeName:}" failed. No retries permitted until 2026-05-28 16:00:50.180606186 +0000 UTC m=+98.740249667 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls") pod "image-registry-78499c94f5-9hjq9" (UID: "796a53ef-2aff-4989-b714-f8643db1c5b8") : secret "image-registry-tls" not found
May 28 16:00:18.281850 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:18.281821    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls\") pod \"dns-default-b2vxs\" (UID: \"a45a3b0c-89c4-4212-967e-a1b96d0c8a81\") " pod="openshift-dns/dns-default-b2vxs"
May 28 16:00:18.282020 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:00:18.281955    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 16:00:18.282083 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:00:18.282022    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls podName:a45a3b0c-89c4-4212-967e-a1b96d0c8a81 nodeName:}" failed. No retries permitted until 2026-05-28 16:00:50.282004392 +0000 UTC m=+98.841647874 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls") pod "dns-default-b2vxs" (UID: "a45a3b0c-89c4-4212-967e-a1b96d0c8a81") : secret "dns-default-metrics-tls" not found
May 28 16:00:20.383874 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:20.383837    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lw6n8" event={"ID":"c73529ee-0f31-4285-85c2-51cdfd913e80","Type":"ContainerStarted","Data":"dd17eba64654f02f65909c840269e984bfdbfe20b146115fdf8431e37cb5debd"}
May 28 16:00:20.384321 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:20.383966    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 16:00:20.399180 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:20.399124    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lw6n8" podStartSLOduration=65.646187636 podStartE2EDuration="1m8.399109382s" podCreationTimestamp="2026-05-28 15:59:12 +0000 UTC" firstStartedPulling="2026-05-28 16:00:16.928440779 +0000 UTC m=+65.488084269" lastFinishedPulling="2026-05-28 16:00:19.681362521 +0000 UTC m=+68.241006015" observedRunningTime="2026-05-28 16:00:20.398869019 +0000 UTC m=+68.958512517" watchObservedRunningTime="2026-05-28 16:00:20.399109382 +0000 UTC m=+68.958752873"
May 28 16:00:50.220007 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:50.219961    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert\") pod \"ingress-canary-ddmdp\" (UID: \"6123b655-5980-4479-9c2a-829cde441719\") " pod="openshift-ingress-canary/ingress-canary-ddmdp"
May 28 16:00:50.220496 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:50.220042    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 16:00:50.220496 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:00:50.220122    2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 16:00:50.220496 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:00:50.220143    2559 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 16:00:50.220496 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:00:50.220154    2559 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78499c94f5-9hjq9: secret "image-registry-tls" not found
May 28 16:00:50.220496 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:00:50.220202    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert podName:6123b655-5980-4479-9c2a-829cde441719 nodeName:}" failed. No retries permitted until 2026-05-28 16:01:54.220182726 +0000 UTC m=+162.779826214 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert") pod "ingress-canary-ddmdp" (UID: "6123b655-5980-4479-9c2a-829cde441719") : secret "canary-serving-cert" not found
May 28 16:00:50.220496 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:00:50.220218    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls podName:796a53ef-2aff-4989-b714-f8643db1c5b8 nodeName:}" failed. No retries permitted until 2026-05-28 16:01:54.220212103 +0000 UTC m=+162.779855581 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls") pod "image-registry-78499c94f5-9hjq9" (UID: "796a53ef-2aff-4989-b714-f8643db1c5b8") : secret "image-registry-tls" not found
May 28 16:00:50.320621 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:50.320579    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls\") pod \"dns-default-b2vxs\" (UID: \"a45a3b0c-89c4-4212-967e-a1b96d0c8a81\") " pod="openshift-dns/dns-default-b2vxs"
May 28 16:00:50.320781 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:00:50.320729    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 16:00:50.320818 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:00:50.320801    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls podName:a45a3b0c-89c4-4212-967e-a1b96d0c8a81 nodeName:}" failed. No retries permitted until 2026-05-28 16:01:54.320785348 +0000 UTC m=+162.880428824 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls") pod "dns-default-b2vxs" (UID: "a45a3b0c-89c4-4212-967e-a1b96d0c8a81") : secret "dns-default-metrics-tls" not found
May 28 16:00:51.388733 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:00:51.388702    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lw6n8"
May 28 16:01:20.852917 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:20.852875    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs\") pod \"network-metrics-daemon-69vjf\" (UID: \"bea98d0f-e54c-4da9-a207-f692c8781bea\") " pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 16:01:20.853356 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:01:20.853014    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 28 16:01:20.853356 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:01:20.853096    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs podName:bea98d0f-e54c-4da9-a207-f692c8781bea nodeName:}" failed. No retries permitted until 2026-05-28 16:03:22.85308065 +0000 UTC m=+251.412724126 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs") pod "network-metrics-daemon-69vjf" (UID: "bea98d0f-e54c-4da9-a207-f692c8781bea") : secret "metrics-daemon-secret" not found
May 28 16:01:21.470716 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:21.470679    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ztbwc_3d8296a6-b347-4d39-a758-d049ee69ef30/dns-node-resolver/0.log"
May 28 16:01:22.070564 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:22.070526    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5dd8s_fd2dd070-87e3-4cda-8fe0-1635aeb20687/node-ca/0.log"
May 28 16:01:49.057443 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.057366    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tfsfh"]
May 28 16:01:49.059539 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.059523    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tfsfh"
May 28 16:01:49.061960 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.061938    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 28 16:01:49.062092 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.061938    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 28 16:01:49.062959 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.062939    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 28 16:01:49.063065 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.062940    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 28 16:01:49.063110 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.062989    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4nwdt\""
May 28 16:01:49.070659 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.070636    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tfsfh"]
May 28 16:01:49.164273 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.164229    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dab12b45-9da4-47e1-8464-f515ac10e266-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tfsfh\" (UID: \"dab12b45-9da4-47e1-8464-f515ac10e266\") " pod="openshift-insights/insights-runtime-extractor-tfsfh"
May 28 16:01:49.164452 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.164385    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xpkf\" (UniqueName: \"kubernetes.io/projected/dab12b45-9da4-47e1-8464-f515ac10e266-kube-api-access-6xpkf\") pod \"insights-runtime-extractor-tfsfh\" (UID: \"dab12b45-9da4-47e1-8464-f515ac10e266\") " pod="openshift-insights/insights-runtime-extractor-tfsfh"
May 28 16:01:49.164452 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.164428    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dab12b45-9da4-47e1-8464-f515ac10e266-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tfsfh\" (UID: \"dab12b45-9da4-47e1-8464-f515ac10e266\") " pod="openshift-insights/insights-runtime-extractor-tfsfh"
May 28 16:01:49.164579 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.164495    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dab12b45-9da4-47e1-8464-f515ac10e266-crio-socket\") pod \"insights-runtime-extractor-tfsfh\" (UID: \"dab12b45-9da4-47e1-8464-f515ac10e266\") " pod="openshift-insights/insights-runtime-extractor-tfsfh"
May 28 16:01:49.164615 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.164580    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dab12b45-9da4-47e1-8464-f515ac10e266-data-volume\") pod \"insights-runtime-extractor-tfsfh\" (UID: \"dab12b45-9da4-47e1-8464-f515ac10e266\") " pod="openshift-insights/insights-runtime-extractor-tfsfh"
May 28 16:01:49.264998 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.264967    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dab12b45-9da4-47e1-8464-f515ac10e266-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tfsfh\" (UID: \"dab12b45-9da4-47e1-8464-f515ac10e266\") " pod="openshift-insights/insights-runtime-extractor-tfsfh"
May 28 16:01:49.264998 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.265017    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dab12b45-9da4-47e1-8464-f515ac10e266-crio-socket\") pod \"insights-runtime-extractor-tfsfh\" (UID: \"dab12b45-9da4-47e1-8464-f515ac10e266\") " pod="openshift-insights/insights-runtime-extractor-tfsfh"
May 28 16:01:49.265245 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.265036    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dab12b45-9da4-47e1-8464-f515ac10e266-data-volume\") pod \"insights-runtime-extractor-tfsfh\" (UID: \"dab12b45-9da4-47e1-8464-f515ac10e266\") " pod="openshift-insights/insights-runtime-extractor-tfsfh"
May 28 16:01:49.265245 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.265125    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dab12b45-9da4-47e1-8464-f515ac10e266-crio-socket\") pod \"insights-runtime-extractor-tfsfh\" (UID: \"dab12b45-9da4-47e1-8464-f515ac10e266\") " pod="openshift-insights/insights-runtime-extractor-tfsfh"
May 28 16:01:49.265245 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.265165    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dab12b45-9da4-47e1-8464-f515ac10e266-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tfsfh\" (UID: \"dab12b45-9da4-47e1-8464-f515ac10e266\") " pod="openshift-insights/insights-runtime-extractor-tfsfh"
May 28 16:01:49.265392 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.265264    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6xpkf\" (UniqueName: \"kubernetes.io/projected/dab12b45-9da4-47e1-8464-f515ac10e266-kube-api-access-6xpkf\") pod \"insights-runtime-extractor-tfsfh\" (UID: \"dab12b45-9da4-47e1-8464-f515ac10e266\") " pod="openshift-insights/insights-runtime-extractor-tfsfh"
May 28 16:01:49.265430 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.265397    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dab12b45-9da4-47e1-8464-f515ac10e266-data-volume\") pod \"insights-runtime-extractor-tfsfh\" (UID: \"dab12b45-9da4-47e1-8464-f515ac10e266\") " pod="openshift-insights/insights-runtime-extractor-tfsfh"
May 28 16:01:49.265614 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.265596    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dab12b45-9da4-47e1-8464-f515ac10e266-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tfsfh\" (UID: \"dab12b45-9da4-47e1-8464-f515ac10e266\") " pod="openshift-insights/insights-runtime-extractor-tfsfh"
May 28 16:01:49.267457 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.267438    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dab12b45-9da4-47e1-8464-f515ac10e266-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tfsfh\" (UID: \"dab12b45-9da4-47e1-8464-f515ac10e266\") " pod="openshift-insights/insights-runtime-extractor-tfsfh"
May 28 16:01:49.273436 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.273406    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xpkf\" (UniqueName: \"kubernetes.io/projected/dab12b45-9da4-47e1-8464-f515ac10e266-kube-api-access-6xpkf\") pod \"insights-runtime-extractor-tfsfh\" (UID: \"dab12b45-9da4-47e1-8464-f515ac10e266\") " pod="openshift-insights/insights-runtime-extractor-tfsfh"
May 28 16:01:49.302778 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:01:49.302734    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-78499c94f5-9hjq9" podUID="796a53ef-2aff-4989-b714-f8643db1c5b8"
May 28 16:01:49.333038 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:01:49.332922    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-ddmdp" podUID="6123b655-5980-4479-9c2a-829cde441719"
May 28 16:01:49.368621 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.368591    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tfsfh"
May 28 16:01:49.375700 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:01:49.375646    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-b2vxs" podUID="a45a3b0c-89c4-4212-967e-a1b96d0c8a81"
May 28 16:01:49.497782 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.497745    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tfsfh"]
May 28 16:01:49.501685 ip-10-0-133-120 kubenswrapper[2559]: W0528 16:01:49.501654    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddab12b45_9da4_47e1_8464_f515ac10e266.slice/crio-d2c587f03ccc6edf2a8b34c741caa5c181b9b22febc6ea3d03ca9e7c581cd6e4 WatchSource:0}: Error finding container d2c587f03ccc6edf2a8b34c741caa5c181b9b22febc6ea3d03ca9e7c581cd6e4: Status 404 returned error can't find the container with id d2c587f03ccc6edf2a8b34c741caa5c181b9b22febc6ea3d03ca9e7c581cd6e4
May 28 16:01:49.591025 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.590929    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tfsfh" event={"ID":"dab12b45-9da4-47e1-8464-f515ac10e266","Type":"ContainerStarted","Data":"f140c2107f21b98a33f75da0ed0e3d76ee64950fd1c9d0a578376ea822643179"}
May 28 16:01:49.591025 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.590970    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tfsfh" event={"ID":"dab12b45-9da4-47e1-8464-f515ac10e266","Type":"ContainerStarted","Data":"d2c587f03ccc6edf2a8b34c741caa5c181b9b22febc6ea3d03ca9e7c581cd6e4"}
May 28 16:01:49.591025 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.590943    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b2vxs"
May 28 16:01:49.591025 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:49.591015    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 16:01:50.091082 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:01:50.091041    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-69vjf" podUID="bea98d0f-e54c-4da9-a207-f692c8781bea"
May 28 16:01:50.594914 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:50.594881    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tfsfh" event={"ID":"dab12b45-9da4-47e1-8464-f515ac10e266","Type":"ContainerStarted","Data":"f293ed1ea7a29f8b3bde8ba2341fb44c9a93199956a7d131586a0d167277686c"}
May 28 16:01:52.601582 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:52.601545    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tfsfh" event={"ID":"dab12b45-9da4-47e1-8464-f515ac10e266","Type":"ContainerStarted","Data":"0d233212f297227d5c15ac3ef6e4976c3a320d08fb09514ba0a40461fff9e2ad"}
May 28 16:01:52.620468 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:52.620415    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tfsfh" podStartSLOduration=1.551508211 podStartE2EDuration="3.620398549s" podCreationTimestamp="2026-05-28 16:01:49 +0000 UTC" firstStartedPulling="2026-05-28 16:01:49.570614657 +0000 UTC m=+158.130258132" lastFinishedPulling="2026-05-28 16:01:51.639504993 +0000 UTC m=+160.199148470" observedRunningTime="2026-05-28 16:01:52.618556059 +0000 UTC m=+161.178199557" watchObservedRunningTime="2026-05-28 16:01:52.620398549 +0000 UTC m=+161.180042053"
May 28 16:01:54.306112 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:54.306078    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 16:01:54.306511 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:54.306152    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert\") pod \"ingress-canary-ddmdp\" (UID: \"6123b655-5980-4479-9c2a-829cde441719\") " pod="openshift-ingress-canary/ingress-canary-ddmdp"
May 28 16:01:54.308507 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:54.308474    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6123b655-5980-4479-9c2a-829cde441719-cert\") pod \"ingress-canary-ddmdp\" (UID: \"6123b655-5980-4479-9c2a-829cde441719\") " pod="openshift-ingress-canary/ingress-canary-ddmdp"
May 28 16:01:54.308616 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:54.308482    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls\") pod \"image-registry-78499c94f5-9hjq9\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") " pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 16:01:54.394524 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:54.394493    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dm5wd\""
May 28 16:01:54.401986 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:54.401966    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 16:01:54.406907 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:54.406881    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls\") pod \"dns-default-b2vxs\" (UID: \"a45a3b0c-89c4-4212-967e-a1b96d0c8a81\") " pod="openshift-dns/dns-default-b2vxs"
May 28 16:01:54.409044 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:54.409027    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a45a3b0c-89c4-4212-967e-a1b96d0c8a81-metrics-tls\") pod \"dns-default-b2vxs\" (UID: \"a45a3b0c-89c4-4212-967e-a1b96d0c8a81\") " pod="openshift-dns/dns-default-b2vxs"
May 28 16:01:54.528877 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:54.528839    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78499c94f5-9hjq9"]
May 28 16:01:54.534054 ip-10-0-133-120 kubenswrapper[2559]: W0528 16:01:54.534024    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod796a53ef_2aff_4989_b714_f8643db1c5b8.slice/crio-56e648f691c1929949065f97593809f954d643e466f86889f689e6f49db94350 WatchSource:0}: Error finding container 56e648f691c1929949065f97593809f954d643e466f86889f689e6f49db94350: Status 404 returned error can't find the container with id 56e648f691c1929949065f97593809f954d643e466f86889f689e6f49db94350
May 28 16:01:54.607233 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:54.607203    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78499c94f5-9hjq9" event={"ID":"796a53ef-2aff-4989-b714-f8643db1c5b8","Type":"ContainerStarted","Data":"56e648f691c1929949065f97593809f954d643e466f86889f689e6f49db94350"}
May 28 16:01:54.695135 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:54.695095    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ptwfd\""
May 28 16:01:54.702461 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:54.702440    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b2vxs"
May 28 16:01:54.830833 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:54.830743    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b2vxs"]
May 28 16:01:54.834590 ip-10-0-133-120 kubenswrapper[2559]: W0528 16:01:54.834561    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda45a3b0c_89c4_4212_967e_a1b96d0c8a81.slice/crio-ae2f3009b0ae8b212b9b44caacf9137f27a6d0fd270c038d08489373a4155b9f WatchSource:0}: Error finding container ae2f3009b0ae8b212b9b44caacf9137f27a6d0fd270c038d08489373a4155b9f: Status 404 returned error can't find the container with id ae2f3009b0ae8b212b9b44caacf9137f27a6d0fd270c038d08489373a4155b9f
May 28 16:01:55.611114 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:55.611068    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78499c94f5-9hjq9" event={"ID":"796a53ef-2aff-4989-b714-f8643db1c5b8","Type":"ContainerStarted","Data":"491f8b2663abae0e2ece9e482d6cea5227ffc2b44786d646025c961f851ca1e0"}
May 28 16:01:55.611611 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:55.611144    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 16:01:55.612186 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:55.612162    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b2vxs" event={"ID":"a45a3b0c-89c4-4212-967e-a1b96d0c8a81","Type":"ContainerStarted","Data":"ae2f3009b0ae8b212b9b44caacf9137f27a6d0fd270c038d08489373a4155b9f"}
May 28 16:01:55.638106 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:55.638059    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-78499c94f5-9hjq9" podStartSLOduration=163.638043926 podStartE2EDuration="2m43.638043926s" podCreationTimestamp="2026-05-28 15:59:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:01:55.636383097 +0000 UTC m=+164.196026596" watchObservedRunningTime="2026-05-28 16:01:55.638043926 +0000 UTC m=+164.197687429"
May 28 16:01:56.619277 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:56.619236    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b2vxs" event={"ID":"a45a3b0c-89c4-4212-967e-a1b96d0c8a81","Type":"ContainerStarted","Data":"1924cf43f1bb1376a5c68ba9912b5cac892adf326cb53d84e71ed81b85c25298"}
May 28 16:01:56.619277 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:56.619276    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b2vxs" event={"ID":"a45a3b0c-89c4-4212-967e-a1b96d0c8a81","Type":"ContainerStarted","Data":"78fdede921ca05063393939148d27443bc81db92612f050cc252b3a539876972"}
May 28 16:01:57.626677 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:57.626646    2559 generic.go:358] "Generic (PLEG): container finished" podID="33449810-e5cd-4fc7-b145-c3ebf5e1618b" containerID="fa1c840c7e02375bb747260753f2bd713d6eccec080444b429ecca9ecd016a44" exitCode=1
May 28 16:01:57.626986 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:57.626719    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c9c4c45fb-v4kbd" event={"ID":"33449810-e5cd-4fc7-b145-c3ebf5e1618b","Type":"ContainerDied","Data":"fa1c840c7e02375bb747260753f2bd713d6eccec080444b429ecca9ecd016a44"}
May 28 16:01:57.626986 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:57.626916    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-b2vxs"
May 28 16:01:57.627104 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:57.627090    2559 scope.go:117] "RemoveContainer" containerID="fa1c840c7e02375bb747260753f2bd713d6eccec080444b429ecca9ecd016a44"
May 28 16:01:57.644002 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:57.643957    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-b2vxs" podStartSLOduration=130.049672092 podStartE2EDuration="2m11.643943346s" podCreationTimestamp="2026-05-28 15:59:46 +0000 UTC" firstStartedPulling="2026-05-28 16:01:54.836356557 +0000 UTC m=+163.396000035" lastFinishedPulling="2026-05-28 16:01:56.430627813 +0000 UTC m=+164.990271289" observedRunningTime="2026-05-28 16:01:56.638590739 +0000 UTC m=+165.198234237" watchObservedRunningTime="2026-05-28 16:01:57.643943346 +0000 UTC m=+166.203586846"
May 28 16:01:58.317533 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:58.317495    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c9c4c45fb-v4kbd"
May 28 16:01:58.631295 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:58.631201    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c9c4c45fb-v4kbd" event={"ID":"33449810-e5cd-4fc7-b145-c3ebf5e1618b","Type":"ContainerStarted","Data":"6c07724d238e805c5326abc5a5d5df50a747a1f4e59219cdf7ad21c6b4560dc4"}
May 28 16:01:58.631722 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:58.631452    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c9c4c45fb-v4kbd"
May 28 16:01:58.632052 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:58.632033    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c9c4c45fb-v4kbd"
May 28 16:01:58.632614 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:58.632594    2559 generic.go:358] "Generic (PLEG): container finished" podID="fc16e686-037b-4387-a421-c5ef14bfc7f5" containerID="ac3b1e624e89f6861a47360d56adf721c297b96fdb8ebe4382a2139f12869ce0" exitCode=255
May 28 16:01:58.632717 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:58.632656    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d97985d-2269t" event={"ID":"fc16e686-037b-4387-a421-c5ef14bfc7f5","Type":"ContainerDied","Data":"ac3b1e624e89f6861a47360d56adf721c297b96fdb8ebe4382a2139f12869ce0"}
May 28 16:01:58.633020 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:58.633004    2559 scope.go:117] "RemoveContainer" containerID="ac3b1e624e89f6861a47360d56adf721c297b96fdb8ebe4382a2139f12869ce0"
May 28 16:01:59.637016 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:01:59.636985    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d97985d-2269t" event={"ID":"fc16e686-037b-4387-a421-c5ef14bfc7f5","Type":"ContainerStarted","Data":"171ef526339ad564498be2980c4aafc70c131d52241ad0fd71472a45e6c06fbb"}
May 28 16:02:01.225063 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.225028    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-58nm8"]
May 28 16:02:01.227495 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.227479    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.230118 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.230097    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 28 16:02:01.230247 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.230193    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 28 16:02:01.230445 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.230430    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 28 16:02:01.230551 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.230534    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 28 16:02:01.231100 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.231083    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 28 16:02:01.231173 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.231125    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 28 16:02:01.231173 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.231148    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-m8pv6\""
May 28 16:02:01.363148 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.363114    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-node-exporter-wtmp\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.363148 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.363152    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-metrics-client-ca\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.363401 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.363170    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.363401 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.363187    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5djw\" (UniqueName: \"kubernetes.io/projected/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-kube-api-access-q5djw\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.363401 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.363280    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-root\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.363401 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.363302    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-node-exporter-textfile\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.363401 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.363353    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-node-exporter-accelerators-collector-config\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.363401 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.363383    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-node-exporter-tls\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.363582 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.363420    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-sys\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.464128 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.464093    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-sys\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.464296 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.464170    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-node-exporter-wtmp\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.464296 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.464215    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-metrics-client-ca\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.464296 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.464241    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.464296 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.464270    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q5djw\" (UniqueName: \"kubernetes.io/projected/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-kube-api-access-q5djw\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.464296 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.464214    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-sys\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.464571 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.464312    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-root\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.464571 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.464363    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-node-exporter-textfile\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.464571 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.464406    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-node-exporter-accelerators-collector-config\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.464571 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.464447    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-node-exporter-tls\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.464571 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.464451    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-root\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.464835 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.464811    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-node-exporter-textfile\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.464882 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.464820    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-node-exporter-wtmp\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.464924 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.464901    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-metrics-client-ca\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.465080 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.465058    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-node-exporter-accelerators-collector-config\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.466748 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.466724    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.466901 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.466885    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-node-exporter-tls\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.484521 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.484497    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5djw\" (UniqueName: \"kubernetes.io/projected/a7fda48a-22aa-4dc4-a60d-ecf919ffc212-kube-api-access-q5djw\") pod \"node-exporter-58nm8\" (UID: \"a7fda48a-22aa-4dc4-a60d-ecf919ffc212\") " pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.536119 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.536072    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-58nm8"
May 28 16:02:01.545132 ip-10-0-133-120 kubenswrapper[2559]: W0528 16:02:01.545094    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7fda48a_22aa_4dc4_a60d_ecf919ffc212.slice/crio-200ce6d0e833f550ced382c5d0e8d0515ef778f7439e692b06f2233f944e7d8d WatchSource:0}: Error finding container 200ce6d0e833f550ced382c5d0e8d0515ef778f7439e692b06f2233f944e7d8d: Status 404 returned error can't find the container with id 200ce6d0e833f550ced382c5d0e8d0515ef778f7439e692b06f2233f944e7d8d
May 28 16:02:01.643291 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:01.643252    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-58nm8" event={"ID":"a7fda48a-22aa-4dc4-a60d-ecf919ffc212","Type":"ContainerStarted","Data":"200ce6d0e833f550ced382c5d0e8d0515ef778f7439e692b06f2233f944e7d8d"}
May 28 16:02:02.646619 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:02.646583    2559 generic.go:358] "Generic (PLEG): container finished" podID="a7fda48a-22aa-4dc4-a60d-ecf919ffc212" containerID="94be1591781443cc02b3c66090bafd12c4df441209f9fa9b83754397b8a55fb3" exitCode=0
May 28 16:02:02.646997 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:02.646671    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-58nm8" event={"ID":"a7fda48a-22aa-4dc4-a60d-ecf919ffc212","Type":"ContainerDied","Data":"94be1591781443cc02b3c66090bafd12c4df441209f9fa9b83754397b8a55fb3"}
May 28 16:02:03.070151 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:03.070109    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ddmdp"
May 28 16:02:03.070461 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:03.070438    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 16:02:03.075857 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:03.075840    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9w77f\""
May 28 16:02:03.081276 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:03.081256    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ddmdp"
May 28 16:02:03.229518 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:03.229485    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ddmdp"]
May 28 16:02:03.232693 ip-10-0-133-120 kubenswrapper[2559]: W0528 16:02:03.232660    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6123b655_5980_4479_9c2a_829cde441719.slice/crio-580cf8feb7ca062faca15d6b5a8f53d54a630e3386a76f908b6002680a6ddd90 WatchSource:0}: Error finding container 580cf8feb7ca062faca15d6b5a8f53d54a630e3386a76f908b6002680a6ddd90: Status 404 returned error can't find the container with id 580cf8feb7ca062faca15d6b5a8f53d54a630e3386a76f908b6002680a6ddd90
May 28 16:02:03.651248 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:03.651209    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-58nm8" event={"ID":"a7fda48a-22aa-4dc4-a60d-ecf919ffc212","Type":"ContainerStarted","Data":"be015c16ea329a17f5f1e3bb935b31f871ed8eac186b0f9a9cb1f2917af81bc3"}
May 28 16:02:03.651719 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:03.651256    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-58nm8" event={"ID":"a7fda48a-22aa-4dc4-a60d-ecf919ffc212","Type":"ContainerStarted","Data":"e3322a5569bc4a38d4d46d102042e4b36007b2db15761981cfa912156f9950ab"}
May 28 16:02:03.652294 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:03.652271    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ddmdp" event={"ID":"6123b655-5980-4479-9c2a-829cde441719","Type":"ContainerStarted","Data":"580cf8feb7ca062faca15d6b5a8f53d54a630e3386a76f908b6002680a6ddd90"}
May 28 16:02:03.681527 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:03.681478    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-58nm8" podStartSLOduration=1.78258114 podStartE2EDuration="2.681459679s" podCreationTimestamp="2026-05-28 16:02:01 +0000 UTC" firstStartedPulling="2026-05-28 16:02:01.547723942 +0000 UTC m=+170.107367419" lastFinishedPulling="2026-05-28 16:02:02.446602478 +0000 UTC m=+171.006245958" observedRunningTime="2026-05-28 16:02:03.679264407 +0000 UTC m=+172.238907905" watchObservedRunningTime="2026-05-28 16:02:03.681459679 +0000 UTC m=+172.241103181"
May 28 16:02:05.658885 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:05.658846    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ddmdp" event={"ID":"6123b655-5980-4479-9c2a-829cde441719","Type":"ContainerStarted","Data":"7bf4ed34ef2e5205a086926e6f34aaeeafca609a7f3bf2d33b8cbfeb6bf6adfb"}
May 28 16:02:05.684238 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:05.684188    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ddmdp" podStartSLOduration=137.728124227 podStartE2EDuration="2m19.684171683s" podCreationTimestamp="2026-05-28 15:59:46 +0000 UTC" firstStartedPulling="2026-05-28 16:02:03.234426797 +0000 UTC m=+171.794070273" lastFinishedPulling="2026-05-28 16:02:05.190474251 +0000 UTC m=+173.750117729" observedRunningTime="2026-05-28 16:02:05.683181332 +0000 UTC m=+174.242824830" watchObservedRunningTime="2026-05-28 16:02:05.684171683 +0000 UTC m=+174.243815180"
May 28 16:02:07.634847 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:07.634819    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-b2vxs"
May 28 16:02:11.626842 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:11.626808    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78499c94f5-9hjq9"]
May 28 16:02:11.631884 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:11.631858    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 16:02:26.609079 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:26.609037    2559 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx" podUID="e717510f-91c8-479f-84be-8ec7b1202691" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 28 16:02:36.609182 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:36.609144    2559 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx" podUID="e717510f-91c8-479f-84be-8ec7b1202691" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 28 16:02:36.645652 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:36.645597    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-78499c94f5-9hjq9" podUID="796a53ef-2aff-4989-b714-f8643db1c5b8" containerName="registry" containerID="cri-o://491f8b2663abae0e2ece9e482d6cea5227ffc2b44786d646025c961f851ca1e0" gracePeriod=30
May 28 16:02:36.887194 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:36.887171    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 16:02:36.938101 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:36.938069    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5t7br\" (UniqueName: \"kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-kube-api-access-5t7br\") pod \"796a53ef-2aff-4989-b714-f8643db1c5b8\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") "
May 28 16:02:36.938277 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:36.938109    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/796a53ef-2aff-4989-b714-f8643db1c5b8-trusted-ca\") pod \"796a53ef-2aff-4989-b714-f8643db1c5b8\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") "
May 28 16:02:36.938277 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:36.938132    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-bound-sa-token\") pod \"796a53ef-2aff-4989-b714-f8643db1c5b8\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") "
May 28 16:02:36.938277 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:36.938159    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/796a53ef-2aff-4989-b714-f8643db1c5b8-image-registry-private-configuration\") pod \"796a53ef-2aff-4989-b714-f8643db1c5b8\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") "
May 28 16:02:36.938277 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:36.938227    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls\") pod \"796a53ef-2aff-4989-b714-f8643db1c5b8\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") "
May 28 16:02:36.938277 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:36.938269    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-certificates\") pod \"796a53ef-2aff-4989-b714-f8643db1c5b8\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") "
May 28 16:02:36.938566 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:36.938299    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/796a53ef-2aff-4989-b714-f8643db1c5b8-ca-trust-extracted\") pod \"796a53ef-2aff-4989-b714-f8643db1c5b8\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") "
May 28 16:02:36.938566 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:36.938359    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/796a53ef-2aff-4989-b714-f8643db1c5b8-installation-pull-secrets\") pod \"796a53ef-2aff-4989-b714-f8643db1c5b8\" (UID: \"796a53ef-2aff-4989-b714-f8643db1c5b8\") "
May 28 16:02:36.938566 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:36.938526    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/796a53ef-2aff-4989-b714-f8643db1c5b8-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "796a53ef-2aff-4989-b714-f8643db1c5b8" (UID: "796a53ef-2aff-4989-b714-f8643db1c5b8"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 16:02:36.938714 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:36.938628    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/796a53ef-2aff-4989-b714-f8643db1c5b8-trusted-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:02:36.938768 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:36.938732    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "796a53ef-2aff-4989-b714-f8643db1c5b8" (UID: "796a53ef-2aff-4989-b714-f8643db1c5b8"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 16:02:36.940641 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:36.940605    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "796a53ef-2aff-4989-b714-f8643db1c5b8" (UID: "796a53ef-2aff-4989-b714-f8643db1c5b8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 16:02:36.940909 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:36.940860    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-kube-api-access-5t7br" (OuterVolumeSpecName: "kube-api-access-5t7br") pod "796a53ef-2aff-4989-b714-f8643db1c5b8" (UID: "796a53ef-2aff-4989-b714-f8643db1c5b8"). InnerVolumeSpecName "kube-api-access-5t7br". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 16:02:36.940909 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:36.940880    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/796a53ef-2aff-4989-b714-f8643db1c5b8-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "796a53ef-2aff-4989-b714-f8643db1c5b8" (UID: "796a53ef-2aff-4989-b714-f8643db1c5b8"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:02:36.941038 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:36.940925    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "796a53ef-2aff-4989-b714-f8643db1c5b8" (UID: "796a53ef-2aff-4989-b714-f8643db1c5b8"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 16:02:36.941038 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:36.940969    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/796a53ef-2aff-4989-b714-f8643db1c5b8-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "796a53ef-2aff-4989-b714-f8643db1c5b8" (UID: "796a53ef-2aff-4989-b714-f8643db1c5b8"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:02:36.949133 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:36.949104    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/796a53ef-2aff-4989-b714-f8643db1c5b8-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "796a53ef-2aff-4989-b714-f8643db1c5b8" (UID: "796a53ef-2aff-4989-b714-f8643db1c5b8"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 16:02:37.039639 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:37.039594    2559 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-certificates\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:02:37.039639 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:37.039632    2559 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/796a53ef-2aff-4989-b714-f8643db1c5b8-ca-trust-extracted\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:02:37.039639 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:37.039645    2559 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/796a53ef-2aff-4989-b714-f8643db1c5b8-installation-pull-secrets\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:02:37.039897 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:37.039657    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5t7br\" (UniqueName: \"kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-kube-api-access-5t7br\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:02:37.039897 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:37.039670    2559 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-bound-sa-token\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:02:37.039897 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:37.039683    2559 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/796a53ef-2aff-4989-b714-f8643db1c5b8-image-registry-private-configuration\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:02:37.039897 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:37.039695    2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/796a53ef-2aff-4989-b714-f8643db1c5b8-registry-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:02:37.736935 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:37.736903    2559 generic.go:358] "Generic (PLEG): container finished" podID="796a53ef-2aff-4989-b714-f8643db1c5b8" containerID="491f8b2663abae0e2ece9e482d6cea5227ffc2b44786d646025c961f851ca1e0" exitCode=0
May 28 16:02:37.737395 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:37.736944    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78499c94f5-9hjq9" event={"ID":"796a53ef-2aff-4989-b714-f8643db1c5b8","Type":"ContainerDied","Data":"491f8b2663abae0e2ece9e482d6cea5227ffc2b44786d646025c961f851ca1e0"}
May 28 16:02:37.737395 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:37.736966    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78499c94f5-9hjq9" event={"ID":"796a53ef-2aff-4989-b714-f8643db1c5b8","Type":"ContainerDied","Data":"56e648f691c1929949065f97593809f954d643e466f86889f689e6f49db94350"}
May 28 16:02:37.737395 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:37.736981    2559 scope.go:117] "RemoveContainer" containerID="491f8b2663abae0e2ece9e482d6cea5227ffc2b44786d646025c961f851ca1e0"
May 28 16:02:37.737395 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:37.736985    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78499c94f5-9hjq9"
May 28 16:02:37.756781 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:37.756766    2559 scope.go:117] "RemoveContainer" containerID="491f8b2663abae0e2ece9e482d6cea5227ffc2b44786d646025c961f851ca1e0"
May 28 16:02:37.757023 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:02:37.757007    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"491f8b2663abae0e2ece9e482d6cea5227ffc2b44786d646025c961f851ca1e0\": container with ID starting with 491f8b2663abae0e2ece9e482d6cea5227ffc2b44786d646025c961f851ca1e0 not found: ID does not exist" containerID="491f8b2663abae0e2ece9e482d6cea5227ffc2b44786d646025c961f851ca1e0"
May 28 16:02:37.757075 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:37.757030    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"491f8b2663abae0e2ece9e482d6cea5227ffc2b44786d646025c961f851ca1e0"} err="failed to get container status \"491f8b2663abae0e2ece9e482d6cea5227ffc2b44786d646025c961f851ca1e0\": rpc error: code = NotFound desc = could not find container \"491f8b2663abae0e2ece9e482d6cea5227ffc2b44786d646025c961f851ca1e0\": container with ID starting with 491f8b2663abae0e2ece9e482d6cea5227ffc2b44786d646025c961f851ca1e0 not found: ID does not exist"
May 28 16:02:37.761777 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:37.760367    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78499c94f5-9hjq9"]
May 28 16:02:37.764146 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:37.764111    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-78499c94f5-9hjq9"]
May 28 16:02:38.073562 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:38.073484    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="796a53ef-2aff-4989-b714-f8643db1c5b8" path="/var/lib/kubelet/pods/796a53ef-2aff-4989-b714-f8643db1c5b8/volumes"
May 28 16:02:46.609047 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:46.609005    2559 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx" podUID="e717510f-91c8-479f-84be-8ec7b1202691" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 28 16:02:46.609424 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:46.609088    2559 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx"
May 28 16:02:46.609580 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:46.609563    2559 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"29fc3a7d61cbdf57512bb5faa32887ba4829a7a8e5eddedb4fede0169e1c9fac"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx" containerMessage="Container service-proxy failed liveness probe, will be restarted"
May 28 16:02:46.609617 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:46.609600    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx" podUID="e717510f-91c8-479f-84be-8ec7b1202691" containerName="service-proxy" containerID="cri-o://29fc3a7d61cbdf57512bb5faa32887ba4829a7a8e5eddedb4fede0169e1c9fac" gracePeriod=30
May 28 16:02:46.762983 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:46.762953    2559 generic.go:358] "Generic (PLEG): container finished" podID="e717510f-91c8-479f-84be-8ec7b1202691" containerID="29fc3a7d61cbdf57512bb5faa32887ba4829a7a8e5eddedb4fede0169e1c9fac" exitCode=2
May 28 16:02:46.763095 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:46.763008    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx" event={"ID":"e717510f-91c8-479f-84be-8ec7b1202691","Type":"ContainerDied","Data":"29fc3a7d61cbdf57512bb5faa32887ba4829a7a8e5eddedb4fede0169e1c9fac"}
May 28 16:02:47.767272 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:02:47.767233    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-556fd7895f-xftpx" event={"ID":"e717510f-91c8-479f-84be-8ec7b1202691","Type":"ContainerStarted","Data":"0303c482a592910143d0f3ed337edb8edaea9b212b9df2b7a3ba24852f64f066"}
May 28 16:03:22.891608 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:03:22.891570    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs\") pod \"network-metrics-daemon-69vjf\" (UID: \"bea98d0f-e54c-4da9-a207-f692c8781bea\") " pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 16:03:22.893859 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:03:22.893838    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bea98d0f-e54c-4da9-a207-f692c8781bea-metrics-certs\") pod \"network-metrics-daemon-69vjf\" (UID: \"bea98d0f-e54c-4da9-a207-f692c8781bea\") " pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 16:03:23.174188 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:03:23.174104    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4ck6s\""
May 28 16:03:23.181995 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:03:23.181977    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69vjf"
May 28 16:03:23.303845 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:03:23.303817    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-69vjf"]
May 28 16:03:23.307241 ip-10-0-133-120 kubenswrapper[2559]: W0528 16:03:23.307209    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbea98d0f_e54c_4da9_a207_f692c8781bea.slice/crio-fb6f58131c1b77c6d12a713aa15104e08f3a99451da42bb601ab74d62c8284fd WatchSource:0}: Error finding container fb6f58131c1b77c6d12a713aa15104e08f3a99451da42bb601ab74d62c8284fd: Status 404 returned error can't find the container with id fb6f58131c1b77c6d12a713aa15104e08f3a99451da42bb601ab74d62c8284fd
May 28 16:03:23.858859 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:03:23.858814    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-69vjf" event={"ID":"bea98d0f-e54c-4da9-a207-f692c8781bea","Type":"ContainerStarted","Data":"fb6f58131c1b77c6d12a713aa15104e08f3a99451da42bb601ab74d62c8284fd"}
May 28 16:03:24.862753 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:03:24.862717    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-69vjf" event={"ID":"bea98d0f-e54c-4da9-a207-f692c8781bea","Type":"ContainerStarted","Data":"7b06ed2f68cf23c7031dba0a3bd6adc453499228ced524c4ca735c3d7bf0ed02"}
May 28 16:03:24.862753 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:03:24.862752    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-69vjf" event={"ID":"bea98d0f-e54c-4da9-a207-f692c8781bea","Type":"ContainerStarted","Data":"ba1693adf9482031739f31f1ea5edb5685cc42e5391296721372799f8c6b5fff"}
May 28 16:03:24.879514 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:03:24.879455    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-69vjf" podStartSLOduration=251.763770272 podStartE2EDuration="4m12.879440723s" podCreationTimestamp="2026-05-28 15:59:12 +0000 UTC" firstStartedPulling="2026-05-28 16:03:23.309371065 +0000 UTC m=+251.869014545" lastFinishedPulling="2026-05-28 16:03:24.425041517 +0000 UTC m=+252.984684996" observedRunningTime="2026-05-28 16:03:24.878318244 +0000 UTC m=+253.437961744" watchObservedRunningTime="2026-05-28 16:03:24.879440723 +0000 UTC m=+253.439084233"
May 28 16:04:11.970253 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:04:11.970223    2559 kubelet.go:1628] "Image garbage collection succeeded"
May 28 16:11:35.673240 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:11:35.673197    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-854544c579-rj5wf"]
May 28 16:11:35.673729 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:11:35.673480    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="796a53ef-2aff-4989-b714-f8643db1c5b8" containerName="registry"
May 28 16:11:35.673729 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:11:35.673493    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="796a53ef-2aff-4989-b714-f8643db1c5b8" containerName="registry"
May 28 16:11:35.673729 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:11:35.673531    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="796a53ef-2aff-4989-b714-f8643db1c5b8" containerName="registry"
May 28 16:11:35.675100 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:11:35.675083    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-854544c579-rj5wf"
May 28 16:11:35.677754 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:11:35.677718    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 28 16:11:35.677754 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:11:35.677732    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-khd9f\""
May 28 16:11:35.678590 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:11:35.678572    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 28 16:11:35.685512 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:11:35.685491    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-854544c579-rj5wf"]
May 28 16:11:35.782309 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:11:35.782262    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhld9\" (UniqueName: \"kubernetes.io/projected/cc359499-3533-445a-bf2a-c58ca8d27c1f-kube-api-access-bhld9\") pod \"redis-854544c579-rj5wf\" (UID: \"cc359499-3533-445a-bf2a-c58ca8d27c1f\") " pod="test-ns-feast/redis-854544c579-rj5wf"
May 28 16:11:35.882679 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:11:35.882647    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bhld9\" (UniqueName: \"kubernetes.io/projected/cc359499-3533-445a-bf2a-c58ca8d27c1f-kube-api-access-bhld9\") pod \"redis-854544c579-rj5wf\" (UID: \"cc359499-3533-445a-bf2a-c58ca8d27c1f\") " pod="test-ns-feast/redis-854544c579-rj5wf"
May 28 16:11:35.890801 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:11:35.890777    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhld9\" (UniqueName: \"kubernetes.io/projected/cc359499-3533-445a-bf2a-c58ca8d27c1f-kube-api-access-bhld9\") pod \"redis-854544c579-rj5wf\" (UID: \"cc359499-3533-445a-bf2a-c58ca8d27c1f\") " pod="test-ns-feast/redis-854544c579-rj5wf"
May 28 16:11:35.984124 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:11:35.984097    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-854544c579-rj5wf"
May 28 16:11:36.109100 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:11:36.109068    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-854544c579-rj5wf"]
May 28 16:11:36.112017 ip-10-0-133-120 kubenswrapper[2559]: W0528 16:11:36.111986    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc359499_3533_445a_bf2a_c58ca8d27c1f.slice/crio-ef342012f1cf943bb9efc6dcfada8a66a12001bb258932572082b0bc7b15e11a WatchSource:0}: Error finding container ef342012f1cf943bb9efc6dcfada8a66a12001bb258932572082b0bc7b15e11a: Status 404 returned error can't find the container with id ef342012f1cf943bb9efc6dcfada8a66a12001bb258932572082b0bc7b15e11a
May 28 16:11:36.114052 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:11:36.114035    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 16:11:37.099441 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:11:37.099392    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-854544c579-rj5wf" event={"ID":"cc359499-3533-445a-bf2a-c58ca8d27c1f","Type":"ContainerStarted","Data":"ef342012f1cf943bb9efc6dcfada8a66a12001bb258932572082b0bc7b15e11a"}
May 28 16:11:38.103463 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:11:38.103425    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-854544c579-rj5wf" event={"ID":"cc359499-3533-445a-bf2a-c58ca8d27c1f","Type":"ContainerStarted","Data":"d2d6da33bd98faac2ffcca9a9aa8430f2f03064796afcc5b9c2706e780e6a59b"}
May 28 16:11:38.117809 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:11:38.117753    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/redis-854544c579-rj5wf" podStartSLOduration=1.684723244 podStartE2EDuration="3.1177345s" podCreationTimestamp="2026-05-28 16:11:35 +0000 UTC" firstStartedPulling="2026-05-28 16:11:36.114184936 +0000 UTC m=+744.673828412" lastFinishedPulling="2026-05-28 16:11:37.547196193 +0000 UTC m=+746.106839668" observedRunningTime="2026-05-28 16:11:38.116907298 +0000 UTC m=+746.676550788" watchObservedRunningTime="2026-05-28 16:11:38.1177345 +0000 UTC m=+746.677377998"
May 28 16:13:13.324810 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:13:13.324758    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-854544c579-rj5wf"]
May 28 16:13:13.325317 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:13:13.324958    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/redis-854544c579-rj5wf" podUID="cc359499-3533-445a-bf2a-c58ca8d27c1f" containerName="redis" containerID="cri-o://d2d6da33bd98faac2ffcca9a9aa8430f2f03064796afcc5b9c2706e780e6a59b" gracePeriod=30
May 28 16:13:13.566120 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:13:13.566093    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-854544c579-rj5wf"
May 28 16:13:13.659719 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:13:13.659637    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhld9\" (UniqueName: \"kubernetes.io/projected/cc359499-3533-445a-bf2a-c58ca8d27c1f-kube-api-access-bhld9\") pod \"cc359499-3533-445a-bf2a-c58ca8d27c1f\" (UID: \"cc359499-3533-445a-bf2a-c58ca8d27c1f\") "
May 28 16:13:13.661801 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:13:13.661776    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc359499-3533-445a-bf2a-c58ca8d27c1f-kube-api-access-bhld9" (OuterVolumeSpecName: "kube-api-access-bhld9") pod "cc359499-3533-445a-bf2a-c58ca8d27c1f" (UID: "cc359499-3533-445a-bf2a-c58ca8d27c1f"). InnerVolumeSpecName "kube-api-access-bhld9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 16:13:13.760998 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:13:13.760961    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhld9\" (UniqueName: \"kubernetes.io/projected/cc359499-3533-445a-bf2a-c58ca8d27c1f-kube-api-access-bhld9\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:13:14.338840 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:13:14.338755    2559 generic.go:358] "Generic (PLEG): container finished" podID="cc359499-3533-445a-bf2a-c58ca8d27c1f" containerID="d2d6da33bd98faac2ffcca9a9aa8430f2f03064796afcc5b9c2706e780e6a59b" exitCode=0
May 28 16:13:14.338840 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:13:14.338807    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-854544c579-rj5wf"
May 28 16:13:14.338840 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:13:14.338816    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-854544c579-rj5wf" event={"ID":"cc359499-3533-445a-bf2a-c58ca8d27c1f","Type":"ContainerDied","Data":"d2d6da33bd98faac2ffcca9a9aa8430f2f03064796afcc5b9c2706e780e6a59b"}
May 28 16:13:14.338840 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:13:14.338837    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-854544c579-rj5wf" event={"ID":"cc359499-3533-445a-bf2a-c58ca8d27c1f","Type":"ContainerDied","Data":"ef342012f1cf943bb9efc6dcfada8a66a12001bb258932572082b0bc7b15e11a"}
May 28 16:13:14.339382 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:13:14.338851    2559 scope.go:117] "RemoveContainer" containerID="d2d6da33bd98faac2ffcca9a9aa8430f2f03064796afcc5b9c2706e780e6a59b"
May 28 16:13:14.346208 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:13:14.346188    2559 scope.go:117] "RemoveContainer" containerID="d2d6da33bd98faac2ffcca9a9aa8430f2f03064796afcc5b9c2706e780e6a59b"
May 28 16:13:14.346495 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:13:14.346475    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2d6da33bd98faac2ffcca9a9aa8430f2f03064796afcc5b9c2706e780e6a59b\": container with ID starting with d2d6da33bd98faac2ffcca9a9aa8430f2f03064796afcc5b9c2706e780e6a59b not found: ID does not exist" containerID="d2d6da33bd98faac2ffcca9a9aa8430f2f03064796afcc5b9c2706e780e6a59b"
May 28 16:13:14.346580 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:13:14.346506    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2d6da33bd98faac2ffcca9a9aa8430f2f03064796afcc5b9c2706e780e6a59b"} err="failed to get container status \"d2d6da33bd98faac2ffcca9a9aa8430f2f03064796afcc5b9c2706e780e6a59b\": rpc error: code = NotFound desc = could not find container \"d2d6da33bd98faac2ffcca9a9aa8430f2f03064796afcc5b9c2706e780e6a59b\": container with ID starting with d2d6da33bd98faac2ffcca9a9aa8430f2f03064796afcc5b9c2706e780e6a59b not found: ID does not exist"
May 28 16:13:14.353690 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:13:14.353669    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-854544c579-rj5wf"]
May 28 16:13:14.356735 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:13:14.356709    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-854544c579-rj5wf"]
May 28 16:13:16.073280 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:13:16.073245    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc359499-3533-445a-bf2a-c58ca8d27c1f" path="/var/lib/kubelet/pods/cc359499-3533-445a-bf2a-c58ca8d27c1f/volumes"
May 28 16:18:05.541056 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:05.541017    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-jmm99"]
May 28 16:18:05.541504 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:05.541257    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc359499-3533-445a-bf2a-c58ca8d27c1f" containerName="redis"
May 28 16:18:05.541504 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:05.541267    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc359499-3533-445a-bf2a-c58ca8d27c1f" containerName="redis"
May 28 16:18:05.541504 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:05.541319    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="cc359499-3533-445a-bf2a-c58ca8d27c1f" containerName="redis"
May 28 16:18:05.544017 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:05.544000    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jmm99"
May 28 16:18:05.546580 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:05.546561    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\""
May 28 16:18:05.547460 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:05.547439    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\""
May 28 16:18:05.547544 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:05.547439    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-w2z7v\""
May 28 16:18:05.559437 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:05.559410    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-jmm99"]
May 28 16:18:05.663583 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:05.663538    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xcpn\" (UniqueName: \"kubernetes.io/projected/6ed866f3-bb47-4d42-bdc9-c389a0e6072a-kube-api-access-7xcpn\") pod \"feast-operator-controller-manager-cf99db75-jmm99\" (UID: \"6ed866f3-bb47-4d42-bdc9-c389a0e6072a\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jmm99"
May 28 16:18:05.764480 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:05.764448    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7xcpn\" (UniqueName: \"kubernetes.io/projected/6ed866f3-bb47-4d42-bdc9-c389a0e6072a-kube-api-access-7xcpn\") pod \"feast-operator-controller-manager-cf99db75-jmm99\" (UID: \"6ed866f3-bb47-4d42-bdc9-c389a0e6072a\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jmm99"
May 28 16:18:05.775716 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:05.775686    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xcpn\" (UniqueName: \"kubernetes.io/projected/6ed866f3-bb47-4d42-bdc9-c389a0e6072a-kube-api-access-7xcpn\") pod \"feast-operator-controller-manager-cf99db75-jmm99\" (UID: \"6ed866f3-bb47-4d42-bdc9-c389a0e6072a\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jmm99"
May 28 16:18:05.853732 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:05.853647    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jmm99"
May 28 16:18:05.974011 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:05.973856    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-jmm99"]
May 28 16:18:05.976666 ip-10-0-133-120 kubenswrapper[2559]: W0528 16:18:05.976628    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ed866f3_bb47_4d42_bdc9_c389a0e6072a.slice/crio-e098dfbc9ecd2aeaacf60bd092102ed00f1ea2a5598adcea778dd07468df3310 WatchSource:0}: Error finding container e098dfbc9ecd2aeaacf60bd092102ed00f1ea2a5598adcea778dd07468df3310: Status 404 returned error can't find the container with id e098dfbc9ecd2aeaacf60bd092102ed00f1ea2a5598adcea778dd07468df3310
May 28 16:18:05.978399 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:05.978383    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 16:18:06.056835 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:06.056805    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jmm99" event={"ID":"6ed866f3-bb47-4d42-bdc9-c389a0e6072a","Type":"ContainerStarted","Data":"e098dfbc9ecd2aeaacf60bd092102ed00f1ea2a5598adcea778dd07468df3310"}
May 28 16:18:09.065581 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:09.065537    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jmm99" event={"ID":"6ed866f3-bb47-4d42-bdc9-c389a0e6072a","Type":"ContainerStarted","Data":"127826ca58d1113ade17b82408f2a94576be728d2a6dd943a47fb4311da5b8ca"}
May 28 16:18:09.066016 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:09.065606    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jmm99"
May 28 16:18:09.084099 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:09.084052    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jmm99" podStartSLOduration=1.379658056 podStartE2EDuration="4.084038821s" podCreationTimestamp="2026-05-28 16:18:05 +0000 UTC" firstStartedPulling="2026-05-28 16:18:05.978508077 +0000 UTC m=+1134.538151557" lastFinishedPulling="2026-05-28 16:18:08.682888839 +0000 UTC m=+1137.242532322" observedRunningTime="2026-05-28 16:18:09.083519743 +0000 UTC m=+1137.643163241" watchObservedRunningTime="2026-05-28 16:18:09.084038821 +0000 UTC m=+1137.643682318"
May 28 16:18:20.073022 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:20.072995    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jmm99"
May 28 16:18:22.317930 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.317894    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"]
May 28 16:18:22.323084 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.323058    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:22.325376 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.325350    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 28 16:18:22.325499 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.325349    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 28 16:18:22.325572 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.325496    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 28 16:18:22.325572 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.325505    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 28 16:18:22.325671 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.325606    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 28 16:18:22.325717 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.325684    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 28 16:18:22.325799 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.325785    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-vzj6j\""
May 28 16:18:22.332400 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.332379    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"]
May 28 16:18:22.381385 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.381345    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4gl8p\" (UID: \"20c3316d-72df-42fc-b349-1b97d63b06f7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:22.381385 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.381387    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gqff\" (UniqueName: \"kubernetes.io/projected/20c3316d-72df-42fc-b349-1b97d63b06f7-kube-api-access-7gqff\") pod \"feast-simple-feast-setup-664fcfcc58-4gl8p\" (UID: \"20c3316d-72df-42fc-b349-1b97d63b06f7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:22.381589 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.381460    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4gl8p\" (UID: \"20c3316d-72df-42fc-b349-1b97d63b06f7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:22.381589 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.381495    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4gl8p\" (UID: \"20c3316d-72df-42fc-b349-1b97d63b06f7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:22.381589 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.381543    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4gl8p\" (UID: \"20c3316d-72df-42fc-b349-1b97d63b06f7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:22.381589 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.381559    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/20c3316d-72df-42fc-b349-1b97d63b06f7-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-4gl8p\" (UID: \"20c3316d-72df-42fc-b349-1b97d63b06f7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:22.482525 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.482489    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4gl8p\" (UID: \"20c3316d-72df-42fc-b349-1b97d63b06f7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:22.482525 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.482528    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4gl8p\" (UID: \"20c3316d-72df-42fc-b349-1b97d63b06f7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:22.482762 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.482573    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4gl8p\" (UID: \"20c3316d-72df-42fc-b349-1b97d63b06f7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:22.482762 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.482597    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/20c3316d-72df-42fc-b349-1b97d63b06f7-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-4gl8p\" (UID: \"20c3316d-72df-42fc-b349-1b97d63b06f7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:22.482762 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.482625    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4gl8p\" (UID: \"20c3316d-72df-42fc-b349-1b97d63b06f7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:22.482762 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.482651    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7gqff\" (UniqueName: \"kubernetes.io/projected/20c3316d-72df-42fc-b349-1b97d63b06f7-kube-api-access-7gqff\") pod \"feast-simple-feast-setup-664fcfcc58-4gl8p\" (UID: \"20c3316d-72df-42fc-b349-1b97d63b06f7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:22.482762 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:18:22.482662    2559 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 28 16:18:22.482762 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:18:22.482755    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-ui-tls podName:20c3316d-72df-42fc-b349-1b97d63b06f7 nodeName:}" failed. No retries permitted until 2026-05-28 16:18:22.982731567 +0000 UTC m=+1151.542375044 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-ui-tls") pod "feast-simple-feast-setup-664fcfcc58-4gl8p" (UID: "20c3316d-72df-42fc-b349-1b97d63b06f7") : secret "feast-simple-feast-setup-ui-tls" not found
May 28 16:18:22.483075 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:18:22.482768    2559 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found
May 28 16:18:22.483075 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:18:22.482834    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-registry-tls podName:20c3316d-72df-42fc-b349-1b97d63b06f7 nodeName:}" failed. No retries permitted until 2026-05-28 16:18:22.98281974 +0000 UTC m=+1151.542463221 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-registry-tls") pod "feast-simple-feast-setup-664fcfcc58-4gl8p" (UID: "20c3316d-72df-42fc-b349-1b97d63b06f7") : secret "feast-simple-feast-setup-registry-tls" not found
May 28 16:18:22.483075 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.483037    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/20c3316d-72df-42fc-b349-1b97d63b06f7-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-4gl8p\" (UID: \"20c3316d-72df-42fc-b349-1b97d63b06f7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:22.485071 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.485050    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4gl8p\" (UID: \"20c3316d-72df-42fc-b349-1b97d63b06f7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:22.485168 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.485094    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4gl8p\" (UID: \"20c3316d-72df-42fc-b349-1b97d63b06f7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:22.490791 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.490769    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gqff\" (UniqueName: \"kubernetes.io/projected/20c3316d-72df-42fc-b349-1b97d63b06f7-kube-api-access-7gqff\") pod \"feast-simple-feast-setup-664fcfcc58-4gl8p\" (UID: \"20c3316d-72df-42fc-b349-1b97d63b06f7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:22.986872 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.986835    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4gl8p\" (UID: \"20c3316d-72df-42fc-b349-1b97d63b06f7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:22.987049 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.986888    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4gl8p\" (UID: \"20c3316d-72df-42fc-b349-1b97d63b06f7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:22.989272 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.989241    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4gl8p\" (UID: \"20c3316d-72df-42fc-b349-1b97d63b06f7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:22.989272 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:22.989258    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4gl8p\" (UID: \"20c3316d-72df-42fc-b349-1b97d63b06f7\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:23.232604 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:23.232565    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:23.362390 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:23.362361    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"]
May 28 16:18:23.364187 ip-10-0-133-120 kubenswrapper[2559]: W0528 16:18:23.364159    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20c3316d_72df_42fc_b349_1b97d63b06f7.slice/crio-961165e706b5c7c7b88765b87e72fd61dc69a3be444e3bf5de5f8e6fc972d1e3 WatchSource:0}: Error finding container 961165e706b5c7c7b88765b87e72fd61dc69a3be444e3bf5de5f8e6fc972d1e3: Status 404 returned error can't find the container with id 961165e706b5c7c7b88765b87e72fd61dc69a3be444e3bf5de5f8e6fc972d1e3
May 28 16:18:24.103911 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:24.103853    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" event={"ID":"20c3316d-72df-42fc-b349-1b97d63b06f7","Type":"ContainerStarted","Data":"961165e706b5c7c7b88765b87e72fd61dc69a3be444e3bf5de5f8e6fc972d1e3"}
May 28 16:18:39.149369 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:39.149317    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" event={"ID":"20c3316d-72df-42fc-b349-1b97d63b06f7","Type":"ContainerStarted","Data":"158eb4206ae9f7a4e763089b8a3cee1c3441004cc12f69fbc73485f832ebda14"}
May 28 16:18:43.160057 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:43.159970    2559 generic.go:358] "Generic (PLEG): container finished" podID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerID="158eb4206ae9f7a4e763089b8a3cee1c3441004cc12f69fbc73485f832ebda14" exitCode=0
May 28 16:18:43.160057 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:43.160042    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" event={"ID":"20c3316d-72df-42fc-b349-1b97d63b06f7","Type":"ContainerDied","Data":"158eb4206ae9f7a4e763089b8a3cee1c3441004cc12f69fbc73485f832ebda14"}
May 28 16:18:44.165359 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:44.165307    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" event={"ID":"20c3316d-72df-42fc-b349-1b97d63b06f7","Type":"ContainerStarted","Data":"af1dc15387581a4b254338c857d3e32d38a3eb9b56ac843ec61c6d03a44d57c2"}
May 28 16:18:49.180025 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:49.179942    2559 generic.go:358] "Generic (PLEG): container finished" podID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerID="af1dc15387581a4b254338c857d3e32d38a3eb9b56ac843ec61c6d03a44d57c2" exitCode=0
May 28 16:18:49.180025 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:49.179995    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" event={"ID":"20c3316d-72df-42fc-b349-1b97d63b06f7","Type":"ContainerDied","Data":"af1dc15387581a4b254338c857d3e32d38a3eb9b56ac843ec61c6d03a44d57c2"}
May 28 16:18:50.186744 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:50.186703    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" event={"ID":"20c3316d-72df-42fc-b349-1b97d63b06f7","Type":"ContainerStarted","Data":"d076a4b1b5eecad70498d6de564e4cb5551ab3d6bdc1be51f0815468dbd7a3ed"}
May 28 16:18:50.187215 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:50.186754    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" event={"ID":"20c3316d-72df-42fc-b349-1b97d63b06f7","Type":"ContainerStarted","Data":"4c25e5a9d6ae42e891ad0b06f095afbc3fa3f88dc990f0326d610d0c9748da2f"}
May 28 16:18:50.187215 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:50.186769    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" event={"ID":"20c3316d-72df-42fc-b349-1b97d63b06f7","Type":"ContainerStarted","Data":"4df2b4cdc803176cc080f695a6407fbb471d4e572465c9b686b81dbeafd2b3d9"}
May 28 16:18:50.187215 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:50.186783    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" event={"ID":"20c3316d-72df-42fc-b349-1b97d63b06f7","Type":"ContainerStarted","Data":"800897af22e56ddad7f6946e8810a09fe43d6b57e483bafd871f3b2b799dd1b5"}
May 28 16:18:50.210434 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:50.210371    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" podStartSLOduration=13.392500894 podStartE2EDuration="28.210351619s" podCreationTimestamp="2026-05-28 16:18:22 +0000 UTC" firstStartedPulling="2026-05-28 16:18:23.365992303 +0000 UTC m=+1151.925635779" lastFinishedPulling="2026-05-28 16:18:38.183843025 +0000 UTC m=+1166.743486504" observedRunningTime="2026-05-28 16:18:50.209276708 +0000 UTC m=+1178.768920207" watchObservedRunningTime="2026-05-28 16:18:50.210351619 +0000 UTC m=+1178.769995119"
May 28 16:18:50.233580 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:50.233527    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:50.233580 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:50.233581    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:50.233845 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:50.233713    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:50.233845 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:50.233742    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:50.235808 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:50.235770    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.15:6571: connect: connection refused"
May 28 16:18:50.235967 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:50.235814    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.15:8443: connect: connection refused"
May 28 16:18:50.235967 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:50.235849    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="online" probeResult="failure" output="Get \"https://10.132.0.15:6567/health\": dial tcp 10.132.0.15:6567: connect: connection refused"
May 28 16:18:50.235967 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:50.235918    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.15:8816: connect: connection refused"
May 28 16:18:53.233760 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:53.233720    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:53.234359 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:53.234285    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:53.234359 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:53.234290    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="online" probeResult="failure" output="Get \"https://10.132.0.15:6567/health\": dial tcp 10.132.0.15:6567: connect: connection refused"
May 28 16:18:53.234359 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:53.234316    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:53.234359 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:53.234353    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:53.234359 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:53.233935    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.15:8816: connect: connection refused"
May 28 16:18:53.234715 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:53.233957    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.15:6571: connect: connection refused"
May 28 16:18:53.234715 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:53.234016    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.15:8443: connect: connection refused"
May 28 16:18:56.234480 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:56.234448    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:56.234969 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:56.234612    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:56.234969 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:56.234843    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:56.235370 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:56.235344    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:56.235476 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:56.235412    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:56.235647 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:56.235632    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:56.239257 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:56.239236    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:18:57.213370 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:18:57.213319    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:19:59.639049 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:19:59.639017    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"]
May 28 16:19:59.639850 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:19:59.639450    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="registry" containerID="cri-o://800897af22e56ddad7f6946e8810a09fe43d6b57e483bafd871f3b2b799dd1b5" gracePeriod=30
May 28 16:19:59.639850 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:19:59.639498    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="offline" containerID="cri-o://4c25e5a9d6ae42e891ad0b06f095afbc3fa3f88dc990f0326d610d0c9748da2f" gracePeriod=30
May 28 16:19:59.639850 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:19:59.639497    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="ui" containerID="cri-o://d076a4b1b5eecad70498d6de564e4cb5551ab3d6bdc1be51f0815468dbd7a3ed" gracePeriod=30
May 28 16:19:59.639850 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:19:59.639536    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="online" containerID="cri-o://4df2b4cdc803176cc080f695a6407fbb471d4e572465c9b686b81dbeafd2b3d9" gracePeriod=30
May 28 16:19:59.946989 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:19:59.946907    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"]
May 28 16:19:59.950429 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:19:59.950407    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:19:59.953549 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:19:59.953524    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-k2wzw\""
May 28 16:19:59.962639 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:19:59.962614    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"]
May 28 16:20:00.106208 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:00.106173    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-7rfn2\" (UID: \"55d96388-8906-435b-827d-d5239d405bdc\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:00.106208 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:00.106212    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx6n8\" (UniqueName: \"kubernetes.io/projected/55d96388-8906-435b-827d-d5239d405bdc-kube-api-access-dx6n8\") pod \"feast-simple-feast-setup-664fcfcc58-7rfn2\" (UID: \"55d96388-8906-435b-827d-d5239d405bdc\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:00.106436 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:00.106258    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-7rfn2\" (UID: \"55d96388-8906-435b-827d-d5239d405bdc\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:00.106436 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:00.106345    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-7rfn2\" (UID: \"55d96388-8906-435b-827d-d5239d405bdc\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:00.106436 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:00.106392    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-7rfn2\" (UID: \"55d96388-8906-435b-827d-d5239d405bdc\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:00.106436 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:00.106420    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/55d96388-8906-435b-827d-d5239d405bdc-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-7rfn2\" (UID: \"55d96388-8906-435b-827d-d5239d405bdc\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:00.207517 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:00.207421    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-7rfn2\" (UID: \"55d96388-8906-435b-827d-d5239d405bdc\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:00.207517 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:00.207496    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-7rfn2\" (UID: \"55d96388-8906-435b-827d-d5239d405bdc\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:00.207713 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:00.207678    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-7rfn2\" (UID: \"55d96388-8906-435b-827d-d5239d405bdc\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:00.207713 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:00.207708    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/55d96388-8906-435b-827d-d5239d405bdc-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-7rfn2\" (UID: \"55d96388-8906-435b-827d-d5239d405bdc\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:00.207799 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:00.207757    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-7rfn2\" (UID: \"55d96388-8906-435b-827d-d5239d405bdc\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:00.207799 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:00.207785    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dx6n8\" (UniqueName: \"kubernetes.io/projected/55d96388-8906-435b-827d-d5239d405bdc-kube-api-access-dx6n8\") pod \"feast-simple-feast-setup-664fcfcc58-7rfn2\" (UID: \"55d96388-8906-435b-827d-d5239d405bdc\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:00.207906 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:20:00.207848    2559 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found
May 28 16:20:00.207960 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:20:00.207919    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-online-tls podName:55d96388-8906-435b-827d-d5239d405bdc nodeName:}" failed. No retries permitted until 2026-05-28 16:20:00.707895588 +0000 UTC m=+1249.267539305 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-online-tls") pod "feast-simple-feast-setup-664fcfcc58-7rfn2" (UID: "55d96388-8906-435b-827d-d5239d405bdc") : secret "feast-simple-feast-setup-online-tls" not found
May 28 16:20:00.208420 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:00.208183    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/55d96388-8906-435b-827d-d5239d405bdc-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-7rfn2\" (UID: \"55d96388-8906-435b-827d-d5239d405bdc\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:00.210180 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:00.210159    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-7rfn2\" (UID: \"55d96388-8906-435b-827d-d5239d405bdc\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:00.210293 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:00.210271    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-7rfn2\" (UID: \"55d96388-8906-435b-827d-d5239d405bdc\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:00.210376 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:00.210360    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-7rfn2\" (UID: \"55d96388-8906-435b-827d-d5239d405bdc\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:00.215465 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:00.215436    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx6n8\" (UniqueName: \"kubernetes.io/projected/55d96388-8906-435b-827d-d5239d405bdc-kube-api-access-dx6n8\") pod \"feast-simple-feast-setup-664fcfcc58-7rfn2\" (UID: \"55d96388-8906-435b-827d-d5239d405bdc\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:00.367784 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:00.367754    2559 generic.go:358] "Generic (PLEG): container finished" podID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerID="d076a4b1b5eecad70498d6de564e4cb5551ab3d6bdc1be51f0815468dbd7a3ed" exitCode=0
May 28 16:20:00.367784 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:00.367781    2559 generic.go:358] "Generic (PLEG): container finished" podID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerID="4df2b4cdc803176cc080f695a6407fbb471d4e572465c9b686b81dbeafd2b3d9" exitCode=0
May 28 16:20:00.367961 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:00.367823    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" event={"ID":"20c3316d-72df-42fc-b349-1b97d63b06f7","Type":"ContainerDied","Data":"d076a4b1b5eecad70498d6de564e4cb5551ab3d6bdc1be51f0815468dbd7a3ed"}
May 28 16:20:00.367961 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:00.367859    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" event={"ID":"20c3316d-72df-42fc-b349-1b97d63b06f7","Type":"ContainerDied","Data":"4df2b4cdc803176cc080f695a6407fbb471d4e572465c9b686b81dbeafd2b3d9"}
May 28 16:20:00.711186 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:00.711151    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-7rfn2\" (UID: \"55d96388-8906-435b-827d-d5239d405bdc\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:00.713542 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:00.713524    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-7rfn2\" (UID: \"55d96388-8906-435b-827d-d5239d405bdc\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:00.860681 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:00.860632    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:00.994420 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:00.994388    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"]
May 28 16:20:00.996403 ip-10-0-133-120 kubenswrapper[2559]: W0528 16:20:00.996373    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55d96388_8906_435b_827d_d5239d405bdc.slice/crio-10fbcba0f0615da04ff44465c5a1191dc6a7619ec1695ffe3d2c7d842fa61138 WatchSource:0}: Error finding container 10fbcba0f0615da04ff44465c5a1191dc6a7619ec1695ffe3d2c7d842fa61138: Status 404 returned error can't find the container with id 10fbcba0f0615da04ff44465c5a1191dc6a7619ec1695ffe3d2c7d842fa61138
May 28 16:20:01.372509 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:01.372387    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" event={"ID":"55d96388-8906-435b-827d-d5239d405bdc","Type":"ContainerStarted","Data":"2ca2f82c83867f1c339c5adf53db41ba2a5aedbcdcb69af5789c045f6bf951e9"}
May 28 16:20:01.372509 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:01.372424    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" event={"ID":"55d96388-8906-435b-827d-d5239d405bdc","Type":"ContainerStarted","Data":"10fbcba0f0615da04ff44465c5a1191dc6a7619ec1695ffe3d2c7d842fa61138"}
May 28 16:20:06.235967 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:06.235917    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.15:8443: connect: connection refused"
May 28 16:20:06.387037 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:06.387006    2559 generic.go:358] "Generic (PLEG): container finished" podID="55d96388-8906-435b-827d-d5239d405bdc" containerID="2ca2f82c83867f1c339c5adf53db41ba2a5aedbcdcb69af5789c045f6bf951e9" exitCode=0
May 28 16:20:06.387217 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:06.387086    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" event={"ID":"55d96388-8906-435b-827d-d5239d405bdc","Type":"ContainerDied","Data":"2ca2f82c83867f1c339c5adf53db41ba2a5aedbcdcb69af5789c045f6bf951e9"}
May 28 16:20:07.211248 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:07.211205    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="online" probeResult="failure" output="Get \"https://10.132.0.15:6567/health\": dial tcp 10.132.0.15:6567: connect: connection refused"
May 28 16:20:07.391368 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:07.391317    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" event={"ID":"55d96388-8906-435b-827d-d5239d405bdc","Type":"ContainerStarted","Data":"55fe24472caa37cebea524da6367e7ce1628cf997515681293ac2d8f59d3b57d"}
May 28 16:20:12.405723 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:12.405688    2559 generic.go:358] "Generic (PLEG): container finished" podID="55d96388-8906-435b-827d-d5239d405bdc" containerID="55fe24472caa37cebea524da6367e7ce1628cf997515681293ac2d8f59d3b57d" exitCode=0
May 28 16:20:12.406096 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:12.405736    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" event={"ID":"55d96388-8906-435b-827d-d5239d405bdc","Type":"ContainerDied","Data":"55fe24472caa37cebea524da6367e7ce1628cf997515681293ac2d8f59d3b57d"}
May 28 16:20:13.412734 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:13.412688    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" event={"ID":"55d96388-8906-435b-827d-d5239d405bdc","Type":"ContainerStarted","Data":"1b6876fbbacdb50fb5b6448418d6ab0b2b94e03b9c59e65218d4db5cf9f0a1d7"}
May 28 16:20:13.413247 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:13.412745    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" event={"ID":"55d96388-8906-435b-827d-d5239d405bdc","Type":"ContainerStarted","Data":"eeb47c1c06f02c412e9fca21674b8fe9764e1f60a8ba20bffe6c697139e48241"}
May 28 16:20:13.413247 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:13.412780    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" event={"ID":"55d96388-8906-435b-827d-d5239d405bdc","Type":"ContainerStarted","Data":"8fa8fd71a0ff3f0e023f8a1831f268d9e86203f774b60feffa028fa3d507a113"}
May 28 16:20:13.413247 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:13.412795    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" event={"ID":"55d96388-8906-435b-827d-d5239d405bdc","Type":"ContainerStarted","Data":"9a37761e4287c5f2dd2e0e9eae7a67c6da41340ab8eb13b23b6c835faaa215a6"}
May 28 16:20:13.436839 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:13.436763    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" podStartSLOduration=14.436743358 podStartE2EDuration="14.436743358s" podCreationTimestamp="2026-05-28 16:19:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:20:13.434651151 +0000 UTC m=+1261.994294655" watchObservedRunningTime="2026-05-28 16:20:13.436743358 +0000 UTC m=+1261.996386858"
May 28 16:20:15.861377 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:15.861307    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:15.861377 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:15.861384    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:15.861931 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:15.861399    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:15.861931 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:15.861411    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:15.863445 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:15.863403    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.16:8816: connect: connection refused"
May 28 16:20:15.863585 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:15.863447    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="online" probeResult="failure" output="Get \"https://10.132.0.16:6567/health\": dial tcp 10.132.0.16:6567: connect: connection refused"
May 28 16:20:15.863585 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:15.863458    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.16:6571: connect: connection refused"
May 28 16:20:15.863740 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:15.863706    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.16:8443: connect: connection refused"
May 28 16:20:16.235795 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:16.235753    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.15:8443: connect: connection refused"
May 28 16:20:17.211155 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:17.211104    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="online" probeResult="failure" output="Get \"https://10.132.0.15:6567/health\": dial tcp 10.132.0.15:6567: connect: connection refused"
May 28 16:20:18.862272 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:18.862243    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:18.862718 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:18.862371    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:18.862773 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:18.862747    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:18.862876 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:18.862854    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:18.862942 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:18.862891    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:18.862942 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:18.862904    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:18.863225 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:18.863206    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:18.863340 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:18.863264    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:18.863340 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:18.863304    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:18.866747 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:18.866727    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:19.431797 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:19.431769    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:19.434663 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:19.434648    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:20:26.236039 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:26.235985    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.15:8443: connect: connection refused"
May 28 16:20:26.236579 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:26.236141    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:20:27.211250 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:27.211196    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="online" probeResult="failure" output="Get \"https://10.132.0.15:6567/health\": dial tcp 10.132.0.15:6567: connect: connection refused"
May 28 16:20:27.211449 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:27.211359    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:20:30.287903 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.287874    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:20:30.349641 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.349602    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/20c3316d-72df-42fc-b349-1b97d63b06f7-feast-data\") pod \"20c3316d-72df-42fc-b349-1b97d63b06f7\" (UID: \"20c3316d-72df-42fc-b349-1b97d63b06f7\") "
May 28 16:20:30.349641 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.349646    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-online-tls\") pod \"20c3316d-72df-42fc-b349-1b97d63b06f7\" (UID: \"20c3316d-72df-42fc-b349-1b97d63b06f7\") "
May 28 16:20:30.349887 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.349673    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-offline-tls\") pod \"20c3316d-72df-42fc-b349-1b97d63b06f7\" (UID: \"20c3316d-72df-42fc-b349-1b97d63b06f7\") "
May 28 16:20:30.349887 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.349710    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7gqff\" (UniqueName: \"kubernetes.io/projected/20c3316d-72df-42fc-b349-1b97d63b06f7-kube-api-access-7gqff\") pod \"20c3316d-72df-42fc-b349-1b97d63b06f7\" (UID: \"20c3316d-72df-42fc-b349-1b97d63b06f7\") "
May 28 16:20:30.349887 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.349756    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-registry-tls\") pod \"20c3316d-72df-42fc-b349-1b97d63b06f7\" (UID: \"20c3316d-72df-42fc-b349-1b97d63b06f7\") "
May 28 16:20:30.349887 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.349842    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-ui-tls\") pod \"20c3316d-72df-42fc-b349-1b97d63b06f7\" (UID: \"20c3316d-72df-42fc-b349-1b97d63b06f7\") "
May 28 16:20:30.350243 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.350202    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20c3316d-72df-42fc-b349-1b97d63b06f7-feast-data" (OuterVolumeSpecName: "feast-data") pod "20c3316d-72df-42fc-b349-1b97d63b06f7" (UID: "20c3316d-72df-42fc-b349-1b97d63b06f7"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 16:20:30.352181 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.352152    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "20c3316d-72df-42fc-b349-1b97d63b06f7" (UID: "20c3316d-72df-42fc-b349-1b97d63b06f7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:20:30.352181 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.352168    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20c3316d-72df-42fc-b349-1b97d63b06f7-kube-api-access-7gqff" (OuterVolumeSpecName: "kube-api-access-7gqff") pod "20c3316d-72df-42fc-b349-1b97d63b06f7" (UID: "20c3316d-72df-42fc-b349-1b97d63b06f7"). InnerVolumeSpecName "kube-api-access-7gqff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 16:20:30.352316 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.352173    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-online-tls" (OuterVolumeSpecName: "online-tls") pod "20c3316d-72df-42fc-b349-1b97d63b06f7" (UID: "20c3316d-72df-42fc-b349-1b97d63b06f7"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:20:30.352316 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.352246    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "20c3316d-72df-42fc-b349-1b97d63b06f7" (UID: "20c3316d-72df-42fc-b349-1b97d63b06f7"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:20:30.352414 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.352311    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "20c3316d-72df-42fc-b349-1b97d63b06f7" (UID: "20c3316d-72df-42fc-b349-1b97d63b06f7"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:20:30.450779 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.450693    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7gqff\" (UniqueName: \"kubernetes.io/projected/20c3316d-72df-42fc-b349-1b97d63b06f7-kube-api-access-7gqff\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:20:30.450779 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.450723    2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-registry-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:20:30.450779 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.450735    2559 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-ui-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:20:30.450779 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.450743    2559 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/20c3316d-72df-42fc-b349-1b97d63b06f7-feast-data\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:20:30.450779 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.450751    2559 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-online-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:20:30.450779 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.450758    2559 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/20c3316d-72df-42fc-b349-1b97d63b06f7-offline-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:20:30.463723 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.463694    2559 generic.go:358] "Generic (PLEG): container finished" podID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerID="4c25e5a9d6ae42e891ad0b06f095afbc3fa3f88dc990f0326d610d0c9748da2f" exitCode=137
May 28 16:20:30.463723 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.463718    2559 generic.go:358] "Generic (PLEG): container finished" podID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerID="800897af22e56ddad7f6946e8810a09fe43d6b57e483bafd871f3b2b799dd1b5" exitCode=137
May 28 16:20:30.463940 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.463774    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"
May 28 16:20:30.463940 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.463783    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" event={"ID":"20c3316d-72df-42fc-b349-1b97d63b06f7","Type":"ContainerDied","Data":"4c25e5a9d6ae42e891ad0b06f095afbc3fa3f88dc990f0326d610d0c9748da2f"}
May 28 16:20:30.463940 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.463821    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" event={"ID":"20c3316d-72df-42fc-b349-1b97d63b06f7","Type":"ContainerDied","Data":"800897af22e56ddad7f6946e8810a09fe43d6b57e483bafd871f3b2b799dd1b5"}
May 28 16:20:30.463940 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.463832    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p" event={"ID":"20c3316d-72df-42fc-b349-1b97d63b06f7","Type":"ContainerDied","Data":"961165e706b5c7c7b88765b87e72fd61dc69a3be444e3bf5de5f8e6fc972d1e3"}
May 28 16:20:30.463940 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.463847    2559 scope.go:117] "RemoveContainer" containerID="d076a4b1b5eecad70498d6de564e4cb5551ab3d6bdc1be51f0815468dbd7a3ed"
May 28 16:20:30.472486 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.472468    2559 scope.go:117] "RemoveContainer" containerID="4c25e5a9d6ae42e891ad0b06f095afbc3fa3f88dc990f0326d610d0c9748da2f"
May 28 16:20:30.481210 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.481186    2559 scope.go:117] "RemoveContainer" containerID="4df2b4cdc803176cc080f695a6407fbb471d4e572465c9b686b81dbeafd2b3d9"
May 28 16:20:30.485527 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.485505    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"]
May 28 16:20:30.488655 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.488635    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-4gl8p"]
May 28 16:20:30.489400 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.489352    2559 scope.go:117] "RemoveContainer" containerID="800897af22e56ddad7f6946e8810a09fe43d6b57e483bafd871f3b2b799dd1b5"
May 28 16:20:30.496114 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.496096    2559 scope.go:117] "RemoveContainer" containerID="af1dc15387581a4b254338c857d3e32d38a3eb9b56ac843ec61c6d03a44d57c2"
May 28 16:20:30.502593 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.502577    2559 scope.go:117] "RemoveContainer" containerID="158eb4206ae9f7a4e763089b8a3cee1c3441004cc12f69fbc73485f832ebda14"
May 28 16:20:30.508837 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.508821    2559 scope.go:117] "RemoveContainer" containerID="d076a4b1b5eecad70498d6de564e4cb5551ab3d6bdc1be51f0815468dbd7a3ed"
May 28 16:20:30.509067 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:20:30.509030    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d076a4b1b5eecad70498d6de564e4cb5551ab3d6bdc1be51f0815468dbd7a3ed\": container with ID starting with d076a4b1b5eecad70498d6de564e4cb5551ab3d6bdc1be51f0815468dbd7a3ed not found: ID does not exist" containerID="d076a4b1b5eecad70498d6de564e4cb5551ab3d6bdc1be51f0815468dbd7a3ed"
May 28 16:20:30.509116 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.509075    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d076a4b1b5eecad70498d6de564e4cb5551ab3d6bdc1be51f0815468dbd7a3ed"} err="failed to get container status \"d076a4b1b5eecad70498d6de564e4cb5551ab3d6bdc1be51f0815468dbd7a3ed\": rpc error: code = NotFound desc = could not find container \"d076a4b1b5eecad70498d6de564e4cb5551ab3d6bdc1be51f0815468dbd7a3ed\": container with ID starting with d076a4b1b5eecad70498d6de564e4cb5551ab3d6bdc1be51f0815468dbd7a3ed not found: ID does not exist"
May 28 16:20:30.509116 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.509096    2559 scope.go:117] "RemoveContainer" containerID="4c25e5a9d6ae42e891ad0b06f095afbc3fa3f88dc990f0326d610d0c9748da2f"
May 28 16:20:30.509314 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:20:30.509297    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c25e5a9d6ae42e891ad0b06f095afbc3fa3f88dc990f0326d610d0c9748da2f\": container with ID starting with 4c25e5a9d6ae42e891ad0b06f095afbc3fa3f88dc990f0326d610d0c9748da2f not found: ID does not exist" containerID="4c25e5a9d6ae42e891ad0b06f095afbc3fa3f88dc990f0326d610d0c9748da2f"
May 28 16:20:30.509393 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.509318    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c25e5a9d6ae42e891ad0b06f095afbc3fa3f88dc990f0326d610d0c9748da2f"} err="failed to get container status \"4c25e5a9d6ae42e891ad0b06f095afbc3fa3f88dc990f0326d610d0c9748da2f\": rpc error: code = NotFound desc = could not find container \"4c25e5a9d6ae42e891ad0b06f095afbc3fa3f88dc990f0326d610d0c9748da2f\": container with ID starting with 4c25e5a9d6ae42e891ad0b06f095afbc3fa3f88dc990f0326d610d0c9748da2f not found: ID does not exist"
May 28 16:20:30.509393 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.509351    2559 scope.go:117] "RemoveContainer" containerID="4df2b4cdc803176cc080f695a6407fbb471d4e572465c9b686b81dbeafd2b3d9"
May 28 16:20:30.509569 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:20:30.509553    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4df2b4cdc803176cc080f695a6407fbb471d4e572465c9b686b81dbeafd2b3d9\": container with ID starting with 4df2b4cdc803176cc080f695a6407fbb471d4e572465c9b686b81dbeafd2b3d9 not found: ID does not exist" containerID="4df2b4cdc803176cc080f695a6407fbb471d4e572465c9b686b81dbeafd2b3d9"
May 28 16:20:30.509607 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.509572    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4df2b4cdc803176cc080f695a6407fbb471d4e572465c9b686b81dbeafd2b3d9"} err="failed to get container status \"4df2b4cdc803176cc080f695a6407fbb471d4e572465c9b686b81dbeafd2b3d9\": rpc error: code = NotFound desc = could not find container \"4df2b4cdc803176cc080f695a6407fbb471d4e572465c9b686b81dbeafd2b3d9\": container with ID starting with 4df2b4cdc803176cc080f695a6407fbb471d4e572465c9b686b81dbeafd2b3d9 not found: ID does not exist"
May 28 16:20:30.509607 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.509585    2559 scope.go:117] "RemoveContainer" containerID="800897af22e56ddad7f6946e8810a09fe43d6b57e483bafd871f3b2b799dd1b5"
May 28 16:20:30.509804 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:20:30.509786    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"800897af22e56ddad7f6946e8810a09fe43d6b57e483bafd871f3b2b799dd1b5\": container with ID starting with 800897af22e56ddad7f6946e8810a09fe43d6b57e483bafd871f3b2b799dd1b5 not found: ID does not exist" containerID="800897af22e56ddad7f6946e8810a09fe43d6b57e483bafd871f3b2b799dd1b5"
May 28 16:20:30.509842 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.509809    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"800897af22e56ddad7f6946e8810a09fe43d6b57e483bafd871f3b2b799dd1b5"} err="failed to get container status \"800897af22e56ddad7f6946e8810a09fe43d6b57e483bafd871f3b2b799dd1b5\": rpc error: code = NotFound desc = could not find container \"800897af22e56ddad7f6946e8810a09fe43d6b57e483bafd871f3b2b799dd1b5\": container with ID starting with 800897af22e56ddad7f6946e8810a09fe43d6b57e483bafd871f3b2b799dd1b5 not found: ID does not exist"
May 28 16:20:30.509842 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.509825    2559 scope.go:117] "RemoveContainer" containerID="af1dc15387581a4b254338c857d3e32d38a3eb9b56ac843ec61c6d03a44d57c2"
May 28 16:20:30.510040 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:20:30.510027    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af1dc15387581a4b254338c857d3e32d38a3eb9b56ac843ec61c6d03a44d57c2\": container with ID starting with af1dc15387581a4b254338c857d3e32d38a3eb9b56ac843ec61c6d03a44d57c2 not found: ID does not exist" containerID="af1dc15387581a4b254338c857d3e32d38a3eb9b56ac843ec61c6d03a44d57c2"
May 28 16:20:30.510078 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.510044    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af1dc15387581a4b254338c857d3e32d38a3eb9b56ac843ec61c6d03a44d57c2"} err="failed to get container status \"af1dc15387581a4b254338c857d3e32d38a3eb9b56ac843ec61c6d03a44d57c2\": rpc error: code = NotFound desc = could not find container \"af1dc15387581a4b254338c857d3e32d38a3eb9b56ac843ec61c6d03a44d57c2\": container with ID starting with af1dc15387581a4b254338c857d3e32d38a3eb9b56ac843ec61c6d03a44d57c2 not found: ID does not exist"
May 28 16:20:30.510078 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.510056    2559 scope.go:117] "RemoveContainer" containerID="158eb4206ae9f7a4e763089b8a3cee1c3441004cc12f69fbc73485f832ebda14"
May 28 16:20:30.510241 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:20:30.510225    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"158eb4206ae9f7a4e763089b8a3cee1c3441004cc12f69fbc73485f832ebda14\": container with ID starting with 158eb4206ae9f7a4e763089b8a3cee1c3441004cc12f69fbc73485f832ebda14 not found: ID does not exist" containerID="158eb4206ae9f7a4e763089b8a3cee1c3441004cc12f69fbc73485f832ebda14"
May 28 16:20:30.510280 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.510246    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"158eb4206ae9f7a4e763089b8a3cee1c3441004cc12f69fbc73485f832ebda14"} err="failed to get container status \"158eb4206ae9f7a4e763089b8a3cee1c3441004cc12f69fbc73485f832ebda14\": rpc error: code = NotFound desc = could not find container \"158eb4206ae9f7a4e763089b8a3cee1c3441004cc12f69fbc73485f832ebda14\": container with ID starting with 158eb4206ae9f7a4e763089b8a3cee1c3441004cc12f69fbc73485f832ebda14 not found: ID does not exist"
May 28 16:20:30.510280 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.510258    2559 scope.go:117] "RemoveContainer" containerID="d076a4b1b5eecad70498d6de564e4cb5551ab3d6bdc1be51f0815468dbd7a3ed"
May 28 16:20:30.510493 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.510474    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d076a4b1b5eecad70498d6de564e4cb5551ab3d6bdc1be51f0815468dbd7a3ed"} err="failed to get container status \"d076a4b1b5eecad70498d6de564e4cb5551ab3d6bdc1be51f0815468dbd7a3ed\": rpc error: code = NotFound desc = could not find container \"d076a4b1b5eecad70498d6de564e4cb5551ab3d6bdc1be51f0815468dbd7a3ed\": container with ID starting with d076a4b1b5eecad70498d6de564e4cb5551ab3d6bdc1be51f0815468dbd7a3ed not found: ID does not exist"
May 28 16:20:30.510544 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.510493    2559 scope.go:117] "RemoveContainer" containerID="4c25e5a9d6ae42e891ad0b06f095afbc3fa3f88dc990f0326d610d0c9748da2f"
May 28 16:20:30.510723 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.510706    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c25e5a9d6ae42e891ad0b06f095afbc3fa3f88dc990f0326d610d0c9748da2f"} err="failed to get container status \"4c25e5a9d6ae42e891ad0b06f095afbc3fa3f88dc990f0326d610d0c9748da2f\": rpc error: code = NotFound desc = could not find container \"4c25e5a9d6ae42e891ad0b06f095afbc3fa3f88dc990f0326d610d0c9748da2f\": container with ID starting with 4c25e5a9d6ae42e891ad0b06f095afbc3fa3f88dc990f0326d610d0c9748da2f not found: ID does not exist"
May 28 16:20:30.510768 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.510724    2559 scope.go:117] "RemoveContainer" containerID="4df2b4cdc803176cc080f695a6407fbb471d4e572465c9b686b81dbeafd2b3d9"
May 28 16:20:30.510914 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.510895    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4df2b4cdc803176cc080f695a6407fbb471d4e572465c9b686b81dbeafd2b3d9"} err="failed to get container status \"4df2b4cdc803176cc080f695a6407fbb471d4e572465c9b686b81dbeafd2b3d9\": rpc error: code = NotFound desc = could not find container \"4df2b4cdc803176cc080f695a6407fbb471d4e572465c9b686b81dbeafd2b3d9\": container with ID starting with 4df2b4cdc803176cc080f695a6407fbb471d4e572465c9b686b81dbeafd2b3d9 not found: ID does not exist"
May 28 16:20:30.510962 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.510914    2559 scope.go:117] "RemoveContainer" containerID="800897af22e56ddad7f6946e8810a09fe43d6b57e483bafd871f3b2b799dd1b5"
May 28 16:20:30.511115 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.511101    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"800897af22e56ddad7f6946e8810a09fe43d6b57e483bafd871f3b2b799dd1b5"} err="failed to get container status \"800897af22e56ddad7f6946e8810a09fe43d6b57e483bafd871f3b2b799dd1b5\": rpc error: code = NotFound desc = could not find container \"800897af22e56ddad7f6946e8810a09fe43d6b57e483bafd871f3b2b799dd1b5\": container with ID starting with 800897af22e56ddad7f6946e8810a09fe43d6b57e483bafd871f3b2b799dd1b5 not found: ID does not exist"
May 28 16:20:30.511152 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.511117    2559 scope.go:117] "RemoveContainer" containerID="af1dc15387581a4b254338c857d3e32d38a3eb9b56ac843ec61c6d03a44d57c2"
May 28 16:20:30.511340 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.511306    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af1dc15387581a4b254338c857d3e32d38a3eb9b56ac843ec61c6d03a44d57c2"} err="failed to get container status \"af1dc15387581a4b254338c857d3e32d38a3eb9b56ac843ec61c6d03a44d57c2\": rpc error: code = NotFound desc = could not find container \"af1dc15387581a4b254338c857d3e32d38a3eb9b56ac843ec61c6d03a44d57c2\": container with ID starting with af1dc15387581a4b254338c857d3e32d38a3eb9b56ac843ec61c6d03a44d57c2 not found: ID does not exist"
May 28 16:20:30.511340 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.511338    2559 scope.go:117] "RemoveContainer" containerID="158eb4206ae9f7a4e763089b8a3cee1c3441004cc12f69fbc73485f832ebda14"
May 28 16:20:30.511528 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:30.511514    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"158eb4206ae9f7a4e763089b8a3cee1c3441004cc12f69fbc73485f832ebda14"} err="failed to get container status \"158eb4206ae9f7a4e763089b8a3cee1c3441004cc12f69fbc73485f832ebda14\": rpc error: code = NotFound desc = could not find container \"158eb4206ae9f7a4e763089b8a3cee1c3441004cc12f69fbc73485f832ebda14\": container with ID starting with 158eb4206ae9f7a4e763089b8a3cee1c3441004cc12f69fbc73485f832ebda14 not found: ID does not exist"
May 28 16:20:32.073875 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:20:32.073840    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" path="/var/lib/kubelet/pods/20c3316d-72df-42fc-b349-1b97d63b06f7/volumes"
May 28 16:21:21.786216 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.786183    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"]
May 28 16:21:21.786827 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.786464    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="feast-init"
May 28 16:21:21.786827 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.786475    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="feast-init"
May 28 16:21:21.786827 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.786484    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="feast-apply"
May 28 16:21:21.786827 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.786490    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="feast-apply"
May 28 16:21:21.786827 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.786499    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="offline"
May 28 16:21:21.786827 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.786504    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="offline"
May 28 16:21:21.786827 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.786512    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="online"
May 28 16:21:21.786827 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.786517    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="online"
May 28 16:21:21.786827 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.786526    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="ui"
May 28 16:21:21.786827 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.786532    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="ui"
May 28 16:21:21.786827 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.786541    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="registry"
May 28 16:21:21.786827 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.786546    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="registry"
May 28 16:21:21.786827 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.786582    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="ui"
May 28 16:21:21.786827 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.786590    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="registry"
May 28 16:21:21.786827 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.786596    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="online"
May 28 16:21:21.786827 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.786603    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="20c3316d-72df-42fc-b349-1b97d63b06f7" containerName="offline"
May 28 16:21:21.791301 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.791281    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:21.793923 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.793899    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 28 16:21:21.794050 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.793901    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 28 16:21:21.794050 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.793901    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 28 16:21:21.794050 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.793936    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-wp5vm\""
May 28 16:21:21.794218 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.794042    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 28 16:21:21.794574 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.794551    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 28 16:21:21.794658 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.794579    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 28 16:21:21.799484 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.799459    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"]
May 28 16:21:21.929281 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.929252    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qfkn\" (UniqueName: \"kubernetes.io/projected/73e02030-e563-4a33-890a-bc6509ff25f3-kube-api-access-2qfkn\") pod \"feast-simple-feast-remote-setup-5cff94746b-bl9ms\" (UID: \"73e02030-e563-4a33-890a-bc6509ff25f3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:21.929456 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.929306    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/73e02030-e563-4a33-890a-bc6509ff25f3-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-bl9ms\" (UID: \"73e02030-e563-4a33-890a-bc6509ff25f3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:21.929456 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.929349    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/73e02030-e563-4a33-890a-bc6509ff25f3-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-bl9ms\" (UID: \"73e02030-e563-4a33-890a-bc6509ff25f3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:21.929456 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.929369    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/73e02030-e563-4a33-890a-bc6509ff25f3-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-bl9ms\" (UID: \"73e02030-e563-4a33-890a-bc6509ff25f3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:21.929456 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.929394    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/73e02030-e563-4a33-890a-bc6509ff25f3-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-bl9ms\" (UID: \"73e02030-e563-4a33-890a-bc6509ff25f3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:21.929456 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:21.929414    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/73e02030-e563-4a33-890a-bc6509ff25f3-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-bl9ms\" (UID: \"73e02030-e563-4a33-890a-bc6509ff25f3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:22.029753 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:22.029720    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2qfkn\" (UniqueName: \"kubernetes.io/projected/73e02030-e563-4a33-890a-bc6509ff25f3-kube-api-access-2qfkn\") pod \"feast-simple-feast-remote-setup-5cff94746b-bl9ms\" (UID: \"73e02030-e563-4a33-890a-bc6509ff25f3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:22.029946 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:22.029767    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/73e02030-e563-4a33-890a-bc6509ff25f3-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-bl9ms\" (UID: \"73e02030-e563-4a33-890a-bc6509ff25f3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:22.029946 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:22.029791    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/73e02030-e563-4a33-890a-bc6509ff25f3-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-bl9ms\" (UID: \"73e02030-e563-4a33-890a-bc6509ff25f3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:22.029946 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:22.029854    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/73e02030-e563-4a33-890a-bc6509ff25f3-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-bl9ms\" (UID: \"73e02030-e563-4a33-890a-bc6509ff25f3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:22.029946 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:22.029912    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/73e02030-e563-4a33-890a-bc6509ff25f3-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-bl9ms\" (UID: \"73e02030-e563-4a33-890a-bc6509ff25f3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:22.029946 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:22.029942    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/73e02030-e563-4a33-890a-bc6509ff25f3-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-bl9ms\" (UID: \"73e02030-e563-4a33-890a-bc6509ff25f3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:22.030266 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:22.030245    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/73e02030-e563-4a33-890a-bc6509ff25f3-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-bl9ms\" (UID: \"73e02030-e563-4a33-890a-bc6509ff25f3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:22.030826 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:22.030801    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/73e02030-e563-4a33-890a-bc6509ff25f3-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-bl9ms\" (UID: \"73e02030-e563-4a33-890a-bc6509ff25f3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:22.032554 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:22.032534    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/73e02030-e563-4a33-890a-bc6509ff25f3-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-bl9ms\" (UID: \"73e02030-e563-4a33-890a-bc6509ff25f3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:22.032644 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:22.032563    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/73e02030-e563-4a33-890a-bc6509ff25f3-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-bl9ms\" (UID: \"73e02030-e563-4a33-890a-bc6509ff25f3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:22.032644 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:22.032635    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/73e02030-e563-4a33-890a-bc6509ff25f3-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-bl9ms\" (UID: \"73e02030-e563-4a33-890a-bc6509ff25f3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:22.039016 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:22.038944    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qfkn\" (UniqueName: \"kubernetes.io/projected/73e02030-e563-4a33-890a-bc6509ff25f3-kube-api-access-2qfkn\") pod \"feast-simple-feast-remote-setup-5cff94746b-bl9ms\" (UID: \"73e02030-e563-4a33-890a-bc6509ff25f3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:22.101288 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:22.101254    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:22.258637 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:22.258582    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"]
May 28 16:21:22.262462 ip-10-0-133-120 kubenswrapper[2559]: W0528 16:21:22.262430    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod73e02030_e563_4a33_890a_bc6509ff25f3.slice/crio-6287d84caf5e58fd7e61474d4bb6e87f701070af827da8b345437fbfc135a612 WatchSource:0}: Error finding container 6287d84caf5e58fd7e61474d4bb6e87f701070af827da8b345437fbfc135a612: Status 404 returned error can't find the container with id 6287d84caf5e58fd7e61474d4bb6e87f701070af827da8b345437fbfc135a612
May 28 16:21:22.605444 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:22.605346    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms" event={"ID":"73e02030-e563-4a33-890a-bc6509ff25f3","Type":"ContainerStarted","Data":"d78ee52619322bd8bc906f98d178735da5ffddce25087af49d8cc98e5dd4409d"}
May 28 16:21:22.605444 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:22.605392    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms" event={"ID":"73e02030-e563-4a33-890a-bc6509ff25f3","Type":"ContainerStarted","Data":"6287d84caf5e58fd7e61474d4bb6e87f701070af827da8b345437fbfc135a612"}
May 28 16:21:27.624121 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:27.624081    2559 generic.go:358] "Generic (PLEG): container finished" podID="73e02030-e563-4a33-890a-bc6509ff25f3" containerID="d78ee52619322bd8bc906f98d178735da5ffddce25087af49d8cc98e5dd4409d" exitCode=0
May 28 16:21:27.624528 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:27.624142    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms" event={"ID":"73e02030-e563-4a33-890a-bc6509ff25f3","Type":"ContainerDied","Data":"d78ee52619322bd8bc906f98d178735da5ffddce25087af49d8cc98e5dd4409d"}
May 28 16:21:28.629716 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:28.629681    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms" event={"ID":"73e02030-e563-4a33-890a-bc6509ff25f3","Type":"ContainerStarted","Data":"5d68af0c2dd10703646d2f98c8682db4dea3a9d530e3bc9394de79fc1977359a"}
May 28 16:21:33.647026 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:33.646941    2559 generic.go:358] "Generic (PLEG): container finished" podID="73e02030-e563-4a33-890a-bc6509ff25f3" containerID="5d68af0c2dd10703646d2f98c8682db4dea3a9d530e3bc9394de79fc1977359a" exitCode=0
May 28 16:21:33.647397 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:33.647017    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms" event={"ID":"73e02030-e563-4a33-890a-bc6509ff25f3","Type":"ContainerDied","Data":"5d68af0c2dd10703646d2f98c8682db4dea3a9d530e3bc9394de79fc1977359a"}
May 28 16:21:34.652781 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:34.652742    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms" event={"ID":"73e02030-e563-4a33-890a-bc6509ff25f3","Type":"ContainerStarted","Data":"dfd05dbd0cc0ccb82b555f97ee5339010afbfbc9ea74cc27c0ad4daeec5f81b3"}
May 28 16:21:34.652781 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:34.652788    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms" event={"ID":"73e02030-e563-4a33-890a-bc6509ff25f3","Type":"ContainerStarted","Data":"279e5b4181c038021505114e1089e6e34591e30bc9e11940123f23d7356a1e1b"}
May 28 16:21:34.653417 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:34.652798    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms" event={"ID":"73e02030-e563-4a33-890a-bc6509ff25f3","Type":"ContainerStarted","Data":"3f267925b76c2afb4276254a870dff934a0d1a28ef2f8b327d765f4171ec169a"}
May 28 16:21:34.677934 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:34.677867    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms" podStartSLOduration=13.677846605 podStartE2EDuration="13.677846605s" podCreationTimestamp="2026-05-28 16:21:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:21:34.677192197 +0000 UTC m=+1343.236835698" watchObservedRunningTime="2026-05-28 16:21:34.677846605 +0000 UTC m=+1343.237490103"
May 28 16:21:37.101911 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:37.101872    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:37.102433 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:37.101996    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:37.102433 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:37.102012    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:37.103674 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:37.103638    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms" podUID="73e02030-e563-4a33-890a-bc6509ff25f3" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.17:8816: connect: connection refused"
May 28 16:21:37.103674 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:37.103656    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms" podUID="73e02030-e563-4a33-890a-bc6509ff25f3" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.17:8443: connect: connection refused"
May 28 16:21:37.103840 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:37.103698    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms" podUID="73e02030-e563-4a33-890a-bc6509ff25f3" containerName="online" probeResult="failure" output="Get \"https://10.132.0.17:6567/health\": dial tcp 10.132.0.17:6567: connect: connection refused"
May 28 16:21:40.102587 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:40.102555    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:40.102985 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:40.102677    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:40.103119 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:40.103099    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:40.103193 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:40.103131    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:40.103468 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:40.103451    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:40.103558 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:40.103503    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:40.106832 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:40.106813    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:40.671282 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:40.671245    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:21:40.674231 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:21:40.674212    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:22:43.048966 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:22:43.048931    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"]
May 28 16:22:43.049598 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:22:43.049404    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms" podUID="73e02030-e563-4a33-890a-bc6509ff25f3" containerName="online" containerID="cri-o://3f267925b76c2afb4276254a870dff934a0d1a28ef2f8b327d765f4171ec169a" gracePeriod=30
May 28 16:22:43.049598 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:22:43.049467    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms" podUID="73e02030-e563-4a33-890a-bc6509ff25f3" containerName="offline" containerID="cri-o://279e5b4181c038021505114e1089e6e34591e30bc9e11940123f23d7356a1e1b" gracePeriod=30
May 28 16:22:43.049781 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:22:43.049730    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms" podUID="73e02030-e563-4a33-890a-bc6509ff25f3" containerName="ui" containerID="cri-o://dfd05dbd0cc0ccb82b555f97ee5339010afbfbc9ea74cc27c0ad4daeec5f81b3" gracePeriod=30
May 28 16:22:43.205719 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:22:43.205686    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"]
May 28 16:22:43.206110 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:22:43.206061    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="registry" containerID="cri-o://9a37761e4287c5f2dd2e0e9eae7a67c6da41340ab8eb13b23b6c835faaa215a6" gracePeriod=30
May 28 16:22:43.206279 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:22:43.206224    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="ui" containerID="cri-o://1b6876fbbacdb50fb5b6448418d6ab0b2b94e03b9c59e65218d4db5cf9f0a1d7" gracePeriod=30
May 28 16:22:43.206466 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:22:43.206137    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="offline" containerID="cri-o://eeb47c1c06f02c412e9fca21674b8fe9764e1f60a8ba20bffe6c697139e48241" gracePeriod=30
May 28 16:22:43.206466 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:22:43.206140    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="online" containerID="cri-o://8fa8fd71a0ff3f0e023f8a1831f268d9e86203f774b60feffa028fa3d507a113" gracePeriod=30
May 28 16:22:43.840888 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:22:43.840856    2559 generic.go:358] "Generic (PLEG): container finished" podID="55d96388-8906-435b-827d-d5239d405bdc" containerID="1b6876fbbacdb50fb5b6448418d6ab0b2b94e03b9c59e65218d4db5cf9f0a1d7" exitCode=0
May 28 16:22:43.841070 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:22:43.840941    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" event={"ID":"55d96388-8906-435b-827d-d5239d405bdc","Type":"ContainerDied","Data":"1b6876fbbacdb50fb5b6448418d6ab0b2b94e03b9c59e65218d4db5cf9f0a1d7"}
May 28 16:22:43.843458 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:22:43.843434    2559 generic.go:358] "Generic (PLEG): container finished" podID="73e02030-e563-4a33-890a-bc6509ff25f3" containerID="dfd05dbd0cc0ccb82b555f97ee5339010afbfbc9ea74cc27c0ad4daeec5f81b3" exitCode=0
May 28 16:22:43.843458 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:22:43.843453    2559 generic.go:358] "Generic (PLEG): container finished" podID="73e02030-e563-4a33-890a-bc6509ff25f3" containerID="3f267925b76c2afb4276254a870dff934a0d1a28ef2f8b327d765f4171ec169a" exitCode=0
May 28 16:22:43.843599 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:22:43.843502    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms" event={"ID":"73e02030-e563-4a33-890a-bc6509ff25f3","Type":"ContainerDied","Data":"dfd05dbd0cc0ccb82b555f97ee5339010afbfbc9ea74cc27c0ad4daeec5f81b3"}
May 28 16:22:43.843599 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:22:43.843526    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms" event={"ID":"73e02030-e563-4a33-890a-bc6509ff25f3","Type":"ContainerDied","Data":"3f267925b76c2afb4276254a870dff934a0d1a28ef2f8b327d765f4171ec169a"}
May 28 16:22:44.848199 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:22:44.848166    2559 generic.go:358] "Generic (PLEG): container finished" podID="55d96388-8906-435b-827d-d5239d405bdc" containerID="8fa8fd71a0ff3f0e023f8a1831f268d9e86203f774b60feffa028fa3d507a113" exitCode=0
May 28 16:22:44.848588 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:22:44.848238    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" event={"ID":"55d96388-8906-435b-827d-d5239d405bdc","Type":"ContainerDied","Data":"8fa8fd71a0ff3f0e023f8a1831f268d9e86203f774b60feffa028fa3d507a113"}
May 28 16:22:48.863709 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:22:48.863660    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.16:8443: connect: connection refused"
May 28 16:22:49.432479 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:22:49.432432    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="online" probeResult="failure" output="Get \"https://10.132.0.16:6567/health\": dial tcp 10.132.0.16:6567: connect: connection refused"
May 28 16:22:50.104088 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:22:50.104041    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms" podUID="73e02030-e563-4a33-890a-bc6509ff25f3" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.17:8443: connect: connection refused"
May 28 16:22:50.671479 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:22:50.671431    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms" podUID="73e02030-e563-4a33-890a-bc6509ff25f3" containerName="online" probeResult="failure" output="Get \"https://10.132.0.17:6567/health\": dial tcp 10.132.0.17:6567: connect: connection refused"
May 28 16:22:58.863385 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:22:58.863307    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.16:8443: connect: connection refused"
May 28 16:22:59.432231 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:22:59.432191    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="online" probeResult="failure" output="Get \"https://10.132.0.16:6567/health\": dial tcp 10.132.0.16:6567: connect: connection refused"
May 28 16:23:00.103278 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:00.103235    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms" podUID="73e02030-e563-4a33-890a-bc6509ff25f3" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.17:8443: connect: connection refused"
May 28 16:23:00.672166 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:00.672116    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms" podUID="73e02030-e563-4a33-890a-bc6509ff25f3" containerName="online" probeResult="failure" output="Get \"https://10.132.0.17:6567/health\": dial tcp 10.132.0.17:6567: connect: connection refused"
May 28 16:23:08.863912 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:08.863867    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.16:8443: connect: connection refused"
May 28 16:23:08.864414 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:08.864031    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:23:09.432926 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:09.432884    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="online" probeResult="failure" output="Get \"https://10.132.0.16:6567/health\": dial tcp 10.132.0.16:6567: connect: connection refused"
May 28 16:23:09.433081 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:09.433014    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:23:10.104111 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:10.104069    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms" podUID="73e02030-e563-4a33-890a-bc6509ff25f3" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.17:8443: connect: connection refused"
May 28 16:23:10.104570 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:10.104197    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:23:10.671765 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:10.671714    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms" podUID="73e02030-e563-4a33-890a-bc6509ff25f3" containerName="online" probeResult="failure" output="Get \"https://10.132.0.17:6567/health\": dial tcp 10.132.0.17:6567: connect: connection refused"
May 28 16:23:10.671940 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:10.671873    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:23:13.699767 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.699745    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:23:13.841412 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.841388    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:23:13.862912 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.862885    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/73e02030-e563-4a33-890a-bc6509ff25f3-feast-data\") pod \"73e02030-e563-4a33-890a-bc6509ff25f3\" (UID: \"73e02030-e563-4a33-890a-bc6509ff25f3\") "
May 28 16:23:13.863106 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.862916    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/73e02030-e563-4a33-890a-bc6509ff25f3-registry-tls\") pod \"73e02030-e563-4a33-890a-bc6509ff25f3\" (UID: \"73e02030-e563-4a33-890a-bc6509ff25f3\") "
May 28 16:23:13.863106 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.862947    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/73e02030-e563-4a33-890a-bc6509ff25f3-online-tls\") pod \"73e02030-e563-4a33-890a-bc6509ff25f3\" (UID: \"73e02030-e563-4a33-890a-bc6509ff25f3\") "
May 28 16:23:13.863106 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.862964    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/73e02030-e563-4a33-890a-bc6509ff25f3-ui-tls\") pod \"73e02030-e563-4a33-890a-bc6509ff25f3\" (UID: \"73e02030-e563-4a33-890a-bc6509ff25f3\") "
May 28 16:23:13.863106 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.863033    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qfkn\" (UniqueName: \"kubernetes.io/projected/73e02030-e563-4a33-890a-bc6509ff25f3-kube-api-access-2qfkn\") pod \"73e02030-e563-4a33-890a-bc6509ff25f3\" (UID: \"73e02030-e563-4a33-890a-bc6509ff25f3\") "
May 28 16:23:13.863106 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.863054    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/73e02030-e563-4a33-890a-bc6509ff25f3-offline-tls\") pod \"73e02030-e563-4a33-890a-bc6509ff25f3\" (UID: \"73e02030-e563-4a33-890a-bc6509ff25f3\") "
May 28 16:23:13.863373 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.863270    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73e02030-e563-4a33-890a-bc6509ff25f3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "73e02030-e563-4a33-890a-bc6509ff25f3" (UID: "73e02030-e563-4a33-890a-bc6509ff25f3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 16:23:13.863448 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.863412    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73e02030-e563-4a33-890a-bc6509ff25f3-feast-data" (OuterVolumeSpecName: "feast-data") pod "73e02030-e563-4a33-890a-bc6509ff25f3" (UID: "73e02030-e563-4a33-890a-bc6509ff25f3"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 16:23:13.865126 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.865096    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73e02030-e563-4a33-890a-bc6509ff25f3-kube-api-access-2qfkn" (OuterVolumeSpecName: "kube-api-access-2qfkn") pod "73e02030-e563-4a33-890a-bc6509ff25f3" (UID: "73e02030-e563-4a33-890a-bc6509ff25f3"). InnerVolumeSpecName "kube-api-access-2qfkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 16:23:13.865232 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.865145    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73e02030-e563-4a33-890a-bc6509ff25f3-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "73e02030-e563-4a33-890a-bc6509ff25f3" (UID: "73e02030-e563-4a33-890a-bc6509ff25f3"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:23:13.865387 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.865359    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73e02030-e563-4a33-890a-bc6509ff25f3-online-tls" (OuterVolumeSpecName: "online-tls") pod "73e02030-e563-4a33-890a-bc6509ff25f3" (UID: "73e02030-e563-4a33-890a-bc6509ff25f3"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:23:13.865476 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.865409    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73e02030-e563-4a33-890a-bc6509ff25f3-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "73e02030-e563-4a33-890a-bc6509ff25f3" (UID: "73e02030-e563-4a33-890a-bc6509ff25f3"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:23:13.921972 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.921938    2559 generic.go:358] "Generic (PLEG): container finished" podID="73e02030-e563-4a33-890a-bc6509ff25f3" containerID="279e5b4181c038021505114e1089e6e34591e30bc9e11940123f23d7356a1e1b" exitCode=137
May 28 16:23:13.922157 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.922005    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"
May 28 16:23:13.922157 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.922019    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms" event={"ID":"73e02030-e563-4a33-890a-bc6509ff25f3","Type":"ContainerDied","Data":"279e5b4181c038021505114e1089e6e34591e30bc9e11940123f23d7356a1e1b"}
May 28 16:23:13.922157 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.922056    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms" event={"ID":"73e02030-e563-4a33-890a-bc6509ff25f3","Type":"ContainerDied","Data":"6287d84caf5e58fd7e61474d4bb6e87f701070af827da8b345437fbfc135a612"}
May 28 16:23:13.922157 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.922072    2559 scope.go:117] "RemoveContainer" containerID="dfd05dbd0cc0ccb82b555f97ee5339010afbfbc9ea74cc27c0ad4daeec5f81b3"
May 28 16:23:13.924678 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.924652    2559 generic.go:358] "Generic (PLEG): container finished" podID="55d96388-8906-435b-827d-d5239d405bdc" containerID="eeb47c1c06f02c412e9fca21674b8fe9764e1f60a8ba20bffe6c697139e48241" exitCode=137
May 28 16:23:13.924678 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.924679    2559 generic.go:358] "Generic (PLEG): container finished" podID="55d96388-8906-435b-827d-d5239d405bdc" containerID="9a37761e4287c5f2dd2e0e9eae7a67c6da41340ab8eb13b23b6c835faaa215a6" exitCode=137
May 28 16:23:13.924835 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.924679    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" event={"ID":"55d96388-8906-435b-827d-d5239d405bdc","Type":"ContainerDied","Data":"eeb47c1c06f02c412e9fca21674b8fe9764e1f60a8ba20bffe6c697139e48241"}
May 28 16:23:13.924835 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.924718    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" event={"ID":"55d96388-8906-435b-827d-d5239d405bdc","Type":"ContainerDied","Data":"9a37761e4287c5f2dd2e0e9eae7a67c6da41340ab8eb13b23b6c835faaa215a6"}
May 28 16:23:13.924835 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.924734    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2" event={"ID":"55d96388-8906-435b-827d-d5239d405bdc","Type":"ContainerDied","Data":"10fbcba0f0615da04ff44465c5a1191dc6a7619ec1695ffe3d2c7d842fa61138"}
May 28 16:23:13.924835 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.924752    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"
May 28 16:23:13.931619 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.931603    2559 scope.go:117] "RemoveContainer" containerID="279e5b4181c038021505114e1089e6e34591e30bc9e11940123f23d7356a1e1b"
May 28 16:23:13.938709 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.938691    2559 scope.go:117] "RemoveContainer" containerID="3f267925b76c2afb4276254a870dff934a0d1a28ef2f8b327d765f4171ec169a"
May 28 16:23:13.942839 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.942818    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"]
May 28 16:23:13.946005 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.945987    2559 scope.go:117] "RemoveContainer" containerID="5d68af0c2dd10703646d2f98c8682db4dea3a9d530e3bc9394de79fc1977359a"
May 28 16:23:13.946609 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.946592    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-bl9ms"]
May 28 16:23:13.952834 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.952814    2559 scope.go:117] "RemoveContainer" containerID="d78ee52619322bd8bc906f98d178735da5ffddce25087af49d8cc98e5dd4409d"
May 28 16:23:13.959195 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.959168    2559 scope.go:117] "RemoveContainer" containerID="dfd05dbd0cc0ccb82b555f97ee5339010afbfbc9ea74cc27c0ad4daeec5f81b3"
May 28 16:23:13.959474 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:23:13.959457    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfd05dbd0cc0ccb82b555f97ee5339010afbfbc9ea74cc27c0ad4daeec5f81b3\": container with ID starting with dfd05dbd0cc0ccb82b555f97ee5339010afbfbc9ea74cc27c0ad4daeec5f81b3 not found: ID does not exist" containerID="dfd05dbd0cc0ccb82b555f97ee5339010afbfbc9ea74cc27c0ad4daeec5f81b3"
May 28 16:23:13.959524 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.959481    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfd05dbd0cc0ccb82b555f97ee5339010afbfbc9ea74cc27c0ad4daeec5f81b3"} err="failed to get container status \"dfd05dbd0cc0ccb82b555f97ee5339010afbfbc9ea74cc27c0ad4daeec5f81b3\": rpc error: code = NotFound desc = could not find container \"dfd05dbd0cc0ccb82b555f97ee5339010afbfbc9ea74cc27c0ad4daeec5f81b3\": container with ID starting with dfd05dbd0cc0ccb82b555f97ee5339010afbfbc9ea74cc27c0ad4daeec5f81b3 not found: ID does not exist"
May 28 16:23:13.959524 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.959502    2559 scope.go:117] "RemoveContainer" containerID="279e5b4181c038021505114e1089e6e34591e30bc9e11940123f23d7356a1e1b"
May 28 16:23:13.959745 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:23:13.959727    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"279e5b4181c038021505114e1089e6e34591e30bc9e11940123f23d7356a1e1b\": container with ID starting with 279e5b4181c038021505114e1089e6e34591e30bc9e11940123f23d7356a1e1b not found: ID does not exist" containerID="279e5b4181c038021505114e1089e6e34591e30bc9e11940123f23d7356a1e1b"
May 28 16:23:13.959793 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.959751    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"279e5b4181c038021505114e1089e6e34591e30bc9e11940123f23d7356a1e1b"} err="failed to get container status \"279e5b4181c038021505114e1089e6e34591e30bc9e11940123f23d7356a1e1b\": rpc error: code = NotFound desc = could not find container \"279e5b4181c038021505114e1089e6e34591e30bc9e11940123f23d7356a1e1b\": container with ID starting with 279e5b4181c038021505114e1089e6e34591e30bc9e11940123f23d7356a1e1b not found: ID does not exist"
May 28 16:23:13.959793 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.959768    2559 scope.go:117] "RemoveContainer" containerID="3f267925b76c2afb4276254a870dff934a0d1a28ef2f8b327d765f4171ec169a"
May 28 16:23:13.959994 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:23:13.959979    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f267925b76c2afb4276254a870dff934a0d1a28ef2f8b327d765f4171ec169a\": container with ID starting with 3f267925b76c2afb4276254a870dff934a0d1a28ef2f8b327d765f4171ec169a not found: ID does not exist" containerID="3f267925b76c2afb4276254a870dff934a0d1a28ef2f8b327d765f4171ec169a"
May 28 16:23:13.960040 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.959998    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f267925b76c2afb4276254a870dff934a0d1a28ef2f8b327d765f4171ec169a"} err="failed to get container status \"3f267925b76c2afb4276254a870dff934a0d1a28ef2f8b327d765f4171ec169a\": rpc error: code = NotFound desc = could not find container \"3f267925b76c2afb4276254a870dff934a0d1a28ef2f8b327d765f4171ec169a\": container with ID starting with 3f267925b76c2afb4276254a870dff934a0d1a28ef2f8b327d765f4171ec169a not found: ID does not exist"
May 28 16:23:13.960040 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.960012    2559 scope.go:117] "RemoveContainer" containerID="5d68af0c2dd10703646d2f98c8682db4dea3a9d530e3bc9394de79fc1977359a"
May 28 16:23:13.960227 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:23:13.960210    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d68af0c2dd10703646d2f98c8682db4dea3a9d530e3bc9394de79fc1977359a\": container with ID starting with 5d68af0c2dd10703646d2f98c8682db4dea3a9d530e3bc9394de79fc1977359a not found: ID does not exist" containerID="5d68af0c2dd10703646d2f98c8682db4dea3a9d530e3bc9394de79fc1977359a"
May 28 16:23:13.960266 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.960232    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d68af0c2dd10703646d2f98c8682db4dea3a9d530e3bc9394de79fc1977359a"} err="failed to get container status \"5d68af0c2dd10703646d2f98c8682db4dea3a9d530e3bc9394de79fc1977359a\": rpc error: code = NotFound desc = could not find container \"5d68af0c2dd10703646d2f98c8682db4dea3a9d530e3bc9394de79fc1977359a\": container with ID starting with 5d68af0c2dd10703646d2f98c8682db4dea3a9d530e3bc9394de79fc1977359a not found: ID does not exist"
May 28 16:23:13.960266 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.960252    2559 scope.go:117] "RemoveContainer" containerID="d78ee52619322bd8bc906f98d178735da5ffddce25087af49d8cc98e5dd4409d"
May 28 16:23:13.960472 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:23:13.960453    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d78ee52619322bd8bc906f98d178735da5ffddce25087af49d8cc98e5dd4409d\": container with ID starting with d78ee52619322bd8bc906f98d178735da5ffddce25087af49d8cc98e5dd4409d not found: ID does not exist" containerID="d78ee52619322bd8bc906f98d178735da5ffddce25087af49d8cc98e5dd4409d"
May 28 16:23:13.960520 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.960476    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d78ee52619322bd8bc906f98d178735da5ffddce25087af49d8cc98e5dd4409d"} err="failed to get container status \"d78ee52619322bd8bc906f98d178735da5ffddce25087af49d8cc98e5dd4409d\": rpc error: code = NotFound desc = could not find container \"d78ee52619322bd8bc906f98d178735da5ffddce25087af49d8cc98e5dd4409d\": container with ID starting with d78ee52619322bd8bc906f98d178735da5ffddce25087af49d8cc98e5dd4409d not found: ID does not exist"
May 28 16:23:13.960520 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.960493    2559 scope.go:117] "RemoveContainer" containerID="1b6876fbbacdb50fb5b6448418d6ab0b2b94e03b9c59e65218d4db5cf9f0a1d7"
May 28 16:23:13.963431 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.963414    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-registry-tls\") pod \"55d96388-8906-435b-827d-d5239d405bdc\" (UID: \"55d96388-8906-435b-827d-d5239d405bdc\") "
May 28 16:23:13.963506 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.963456    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-offline-tls\") pod \"55d96388-8906-435b-827d-d5239d405bdc\" (UID: \"55d96388-8906-435b-827d-d5239d405bdc\") "
May 28 16:23:13.963506 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.963479    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-ui-tls\") pod \"55d96388-8906-435b-827d-d5239d405bdc\" (UID: \"55d96388-8906-435b-827d-d5239d405bdc\") "
May 28 16:23:13.963506 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.963504    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-online-tls\") pod \"55d96388-8906-435b-827d-d5239d405bdc\" (UID: \"55d96388-8906-435b-827d-d5239d405bdc\") "
May 28 16:23:13.963626 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.963542    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/55d96388-8906-435b-827d-d5239d405bdc-feast-data\") pod \"55d96388-8906-435b-827d-d5239d405bdc\" (UID: \"55d96388-8906-435b-827d-d5239d405bdc\") "
May 28 16:23:13.963626 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.963572    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dx6n8\" (UniqueName: \"kubernetes.io/projected/55d96388-8906-435b-827d-d5239d405bdc-kube-api-access-dx6n8\") pod \"55d96388-8906-435b-827d-d5239d405bdc\" (UID: \"55d96388-8906-435b-827d-d5239d405bdc\") "
May 28 16:23:13.963782 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.963759    2559 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/73e02030-e563-4a33-890a-bc6509ff25f3-feast-data\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:23:13.963782 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.963782    2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/73e02030-e563-4a33-890a-bc6509ff25f3-registry-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:23:13.963956 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.963796    2559 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/73e02030-e563-4a33-890a-bc6509ff25f3-online-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:23:13.963956 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.963809    2559 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/73e02030-e563-4a33-890a-bc6509ff25f3-ui-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:23:13.963956 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.963822    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qfkn\" (UniqueName: \"kubernetes.io/projected/73e02030-e563-4a33-890a-bc6509ff25f3-kube-api-access-2qfkn\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:23:13.963956 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.963834    2559 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/73e02030-e563-4a33-890a-bc6509ff25f3-offline-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:23:13.964224 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.964181    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55d96388-8906-435b-827d-d5239d405bdc-feast-data" (OuterVolumeSpecName: "feast-data") pod "55d96388-8906-435b-827d-d5239d405bdc" (UID: "55d96388-8906-435b-827d-d5239d405bdc"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 16:23:13.965669 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.965644    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "55d96388-8906-435b-827d-d5239d405bdc" (UID: "55d96388-8906-435b-827d-d5239d405bdc"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:23:13.965988 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.965965    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "55d96388-8906-435b-827d-d5239d405bdc" (UID: "55d96388-8906-435b-827d-d5239d405bdc"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:23:13.966082 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.965987    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "55d96388-8906-435b-827d-d5239d405bdc" (UID: "55d96388-8906-435b-827d-d5239d405bdc"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:23:13.966082 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.966002    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-online-tls" (OuterVolumeSpecName: "online-tls") pod "55d96388-8906-435b-827d-d5239d405bdc" (UID: "55d96388-8906-435b-827d-d5239d405bdc"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:23:13.966169 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.966099    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55d96388-8906-435b-827d-d5239d405bdc-kube-api-access-dx6n8" (OuterVolumeSpecName: "kube-api-access-dx6n8") pod "55d96388-8906-435b-827d-d5239d405bdc" (UID: "55d96388-8906-435b-827d-d5239d405bdc"). InnerVolumeSpecName "kube-api-access-dx6n8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 16:23:13.968412 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.968394    2559 scope.go:117] "RemoveContainer" containerID="eeb47c1c06f02c412e9fca21674b8fe9764e1f60a8ba20bffe6c697139e48241"
May 28 16:23:13.975185 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.975165    2559 scope.go:117] "RemoveContainer" containerID="8fa8fd71a0ff3f0e023f8a1831f268d9e86203f774b60feffa028fa3d507a113"
May 28 16:23:13.981617 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.981601    2559 scope.go:117] "RemoveContainer" containerID="9a37761e4287c5f2dd2e0e9eae7a67c6da41340ab8eb13b23b6c835faaa215a6"
May 28 16:23:13.988218 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.988194    2559 scope.go:117] "RemoveContainer" containerID="55fe24472caa37cebea524da6367e7ce1628cf997515681293ac2d8f59d3b57d"
May 28 16:23:13.994865 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:13.994840    2559 scope.go:117] "RemoveContainer" containerID="2ca2f82c83867f1c339c5adf53db41ba2a5aedbcdcb69af5789c045f6bf951e9"
May 28 16:23:14.001227 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.001210    2559 scope.go:117] "RemoveContainer" containerID="1b6876fbbacdb50fb5b6448418d6ab0b2b94e03b9c59e65218d4db5cf9f0a1d7"
May 28 16:23:14.001503 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:23:14.001486    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b6876fbbacdb50fb5b6448418d6ab0b2b94e03b9c59e65218d4db5cf9f0a1d7\": container with ID starting with 1b6876fbbacdb50fb5b6448418d6ab0b2b94e03b9c59e65218d4db5cf9f0a1d7 not found: ID does not exist" containerID="1b6876fbbacdb50fb5b6448418d6ab0b2b94e03b9c59e65218d4db5cf9f0a1d7"
May 28 16:23:14.001577 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.001516    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b6876fbbacdb50fb5b6448418d6ab0b2b94e03b9c59e65218d4db5cf9f0a1d7"} err="failed to get container status \"1b6876fbbacdb50fb5b6448418d6ab0b2b94e03b9c59e65218d4db5cf9f0a1d7\": rpc error: code = NotFound desc = could not find container \"1b6876fbbacdb50fb5b6448418d6ab0b2b94e03b9c59e65218d4db5cf9f0a1d7\": container with ID starting with 1b6876fbbacdb50fb5b6448418d6ab0b2b94e03b9c59e65218d4db5cf9f0a1d7 not found: ID does not exist"
May 28 16:23:14.001577 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.001544    2559 scope.go:117] "RemoveContainer" containerID="eeb47c1c06f02c412e9fca21674b8fe9764e1f60a8ba20bffe6c697139e48241"
May 28 16:23:14.001770 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:23:14.001752    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eeb47c1c06f02c412e9fca21674b8fe9764e1f60a8ba20bffe6c697139e48241\": container with ID starting with eeb47c1c06f02c412e9fca21674b8fe9764e1f60a8ba20bffe6c697139e48241 not found: ID does not exist" containerID="eeb47c1c06f02c412e9fca21674b8fe9764e1f60a8ba20bffe6c697139e48241"
May 28 16:23:14.001811 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.001775    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eeb47c1c06f02c412e9fca21674b8fe9764e1f60a8ba20bffe6c697139e48241"} err="failed to get container status \"eeb47c1c06f02c412e9fca21674b8fe9764e1f60a8ba20bffe6c697139e48241\": rpc error: code = NotFound desc = could not find container \"eeb47c1c06f02c412e9fca21674b8fe9764e1f60a8ba20bffe6c697139e48241\": container with ID starting with eeb47c1c06f02c412e9fca21674b8fe9764e1f60a8ba20bffe6c697139e48241 not found: ID does not exist"
May 28 16:23:14.001811 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.001794    2559 scope.go:117] "RemoveContainer" containerID="8fa8fd71a0ff3f0e023f8a1831f268d9e86203f774b60feffa028fa3d507a113"
May 28 16:23:14.002003 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:23:14.001986    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fa8fd71a0ff3f0e023f8a1831f268d9e86203f774b60feffa028fa3d507a113\": container with ID starting with 8fa8fd71a0ff3f0e023f8a1831f268d9e86203f774b60feffa028fa3d507a113 not found: ID does not exist" containerID="8fa8fd71a0ff3f0e023f8a1831f268d9e86203f774b60feffa028fa3d507a113"
May 28 16:23:14.002041 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.002011    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fa8fd71a0ff3f0e023f8a1831f268d9e86203f774b60feffa028fa3d507a113"} err="failed to get container status \"8fa8fd71a0ff3f0e023f8a1831f268d9e86203f774b60feffa028fa3d507a113\": rpc error: code = NotFound desc = could not find container \"8fa8fd71a0ff3f0e023f8a1831f268d9e86203f774b60feffa028fa3d507a113\": container with ID starting with 8fa8fd71a0ff3f0e023f8a1831f268d9e86203f774b60feffa028fa3d507a113 not found: ID does not exist"
May 28 16:23:14.002041 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.002028    2559 scope.go:117] "RemoveContainer" containerID="9a37761e4287c5f2dd2e0e9eae7a67c6da41340ab8eb13b23b6c835faaa215a6"
May 28 16:23:14.002254 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:23:14.002239    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a37761e4287c5f2dd2e0e9eae7a67c6da41340ab8eb13b23b6c835faaa215a6\": container with ID starting with 9a37761e4287c5f2dd2e0e9eae7a67c6da41340ab8eb13b23b6c835faaa215a6 not found: ID does not exist" containerID="9a37761e4287c5f2dd2e0e9eae7a67c6da41340ab8eb13b23b6c835faaa215a6"
May 28 16:23:14.002317 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.002261    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a37761e4287c5f2dd2e0e9eae7a67c6da41340ab8eb13b23b6c835faaa215a6"} err="failed to get container status \"9a37761e4287c5f2dd2e0e9eae7a67c6da41340ab8eb13b23b6c835faaa215a6\": rpc error: code = NotFound desc = could not find container \"9a37761e4287c5f2dd2e0e9eae7a67c6da41340ab8eb13b23b6c835faaa215a6\": container with ID starting with 9a37761e4287c5f2dd2e0e9eae7a67c6da41340ab8eb13b23b6c835faaa215a6 not found: ID does not exist"
May 28 16:23:14.002317 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.002287    2559 scope.go:117] "RemoveContainer" containerID="55fe24472caa37cebea524da6367e7ce1628cf997515681293ac2d8f59d3b57d"
May 28 16:23:14.002536 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:23:14.002515    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55fe24472caa37cebea524da6367e7ce1628cf997515681293ac2d8f59d3b57d\": container with ID starting with 55fe24472caa37cebea524da6367e7ce1628cf997515681293ac2d8f59d3b57d not found: ID does not exist" containerID="55fe24472caa37cebea524da6367e7ce1628cf997515681293ac2d8f59d3b57d"
May 28 16:23:14.002577 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.002544    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55fe24472caa37cebea524da6367e7ce1628cf997515681293ac2d8f59d3b57d"} err="failed to get container status \"55fe24472caa37cebea524da6367e7ce1628cf997515681293ac2d8f59d3b57d\": rpc error: code = NotFound desc = could not find container \"55fe24472caa37cebea524da6367e7ce1628cf997515681293ac2d8f59d3b57d\": container with ID starting with 55fe24472caa37cebea524da6367e7ce1628cf997515681293ac2d8f59d3b57d not found: ID does not exist"
May 28 16:23:14.002577 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.002560    2559 scope.go:117] "RemoveContainer" containerID="2ca2f82c83867f1c339c5adf53db41ba2a5aedbcdcb69af5789c045f6bf951e9"
May 28 16:23:14.002761 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:23:14.002743    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ca2f82c83867f1c339c5adf53db41ba2a5aedbcdcb69af5789c045f6bf951e9\": container with ID starting with 2ca2f82c83867f1c339c5adf53db41ba2a5aedbcdcb69af5789c045f6bf951e9 not found: ID does not exist" containerID="2ca2f82c83867f1c339c5adf53db41ba2a5aedbcdcb69af5789c045f6bf951e9"
May 28 16:23:14.002803 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.002769    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ca2f82c83867f1c339c5adf53db41ba2a5aedbcdcb69af5789c045f6bf951e9"} err="failed to get container status \"2ca2f82c83867f1c339c5adf53db41ba2a5aedbcdcb69af5789c045f6bf951e9\": rpc error: code = NotFound desc = could not find container \"2ca2f82c83867f1c339c5adf53db41ba2a5aedbcdcb69af5789c045f6bf951e9\": container with ID starting with 2ca2f82c83867f1c339c5adf53db41ba2a5aedbcdcb69af5789c045f6bf951e9 not found: ID does not exist"
May 28 16:23:14.002803 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.002785    2559 scope.go:117] "RemoveContainer" containerID="1b6876fbbacdb50fb5b6448418d6ab0b2b94e03b9c59e65218d4db5cf9f0a1d7"
May 28 16:23:14.003005 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.002988    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b6876fbbacdb50fb5b6448418d6ab0b2b94e03b9c59e65218d4db5cf9f0a1d7"} err="failed to get container status \"1b6876fbbacdb50fb5b6448418d6ab0b2b94e03b9c59e65218d4db5cf9f0a1d7\": rpc error: code = NotFound desc = could not find container \"1b6876fbbacdb50fb5b6448418d6ab0b2b94e03b9c59e65218d4db5cf9f0a1d7\": container with ID starting with 1b6876fbbacdb50fb5b6448418d6ab0b2b94e03b9c59e65218d4db5cf9f0a1d7 not found: ID does not exist"
May 28 16:23:14.003075 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.003007    2559 scope.go:117] "RemoveContainer" containerID="eeb47c1c06f02c412e9fca21674b8fe9764e1f60a8ba20bffe6c697139e48241"
May 28 16:23:14.003237 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.003220    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eeb47c1c06f02c412e9fca21674b8fe9764e1f60a8ba20bffe6c697139e48241"} err="failed to get container status \"eeb47c1c06f02c412e9fca21674b8fe9764e1f60a8ba20bffe6c697139e48241\": rpc error: code = NotFound desc = could not find container \"eeb47c1c06f02c412e9fca21674b8fe9764e1f60a8ba20bffe6c697139e48241\": container with ID starting with eeb47c1c06f02c412e9fca21674b8fe9764e1f60a8ba20bffe6c697139e48241 not found: ID does not exist"
May 28 16:23:14.003287 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.003237    2559 scope.go:117] "RemoveContainer" containerID="8fa8fd71a0ff3f0e023f8a1831f268d9e86203f774b60feffa028fa3d507a113"
May 28 16:23:14.003497 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.003480    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fa8fd71a0ff3f0e023f8a1831f268d9e86203f774b60feffa028fa3d507a113"} err="failed to get container status \"8fa8fd71a0ff3f0e023f8a1831f268d9e86203f774b60feffa028fa3d507a113\": rpc error: code = NotFound desc = could not find container \"8fa8fd71a0ff3f0e023f8a1831f268d9e86203f774b60feffa028fa3d507a113\": container with ID starting with 8fa8fd71a0ff3f0e023f8a1831f268d9e86203f774b60feffa028fa3d507a113 not found: ID does not exist"
May 28 16:23:14.003547 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.003499    2559 scope.go:117] "RemoveContainer" containerID="9a37761e4287c5f2dd2e0e9eae7a67c6da41340ab8eb13b23b6c835faaa215a6"
May 28 16:23:14.003710 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.003692    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a37761e4287c5f2dd2e0e9eae7a67c6da41340ab8eb13b23b6c835faaa215a6"} err="failed to get container status \"9a37761e4287c5f2dd2e0e9eae7a67c6da41340ab8eb13b23b6c835faaa215a6\": rpc error: code = NotFound desc = could not find container \"9a37761e4287c5f2dd2e0e9eae7a67c6da41340ab8eb13b23b6c835faaa215a6\": container with ID starting with 9a37761e4287c5f2dd2e0e9eae7a67c6da41340ab8eb13b23b6c835faaa215a6 not found: ID does not exist"
May 28 16:23:14.003775 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.003713    2559 scope.go:117] "RemoveContainer" containerID="55fe24472caa37cebea524da6367e7ce1628cf997515681293ac2d8f59d3b57d"
May 28 16:23:14.003912 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.003897    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55fe24472caa37cebea524da6367e7ce1628cf997515681293ac2d8f59d3b57d"} err="failed to get container status \"55fe24472caa37cebea524da6367e7ce1628cf997515681293ac2d8f59d3b57d\": rpc error: code = NotFound desc = could not find container \"55fe24472caa37cebea524da6367e7ce1628cf997515681293ac2d8f59d3b57d\": container with ID starting with 55fe24472caa37cebea524da6367e7ce1628cf997515681293ac2d8f59d3b57d not found: ID does not exist"
May 28 16:23:14.003962 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.003912    2559 scope.go:117] "RemoveContainer" containerID="2ca2f82c83867f1c339c5adf53db41ba2a5aedbcdcb69af5789c045f6bf951e9"
May 28 16:23:14.004086 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.004070    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ca2f82c83867f1c339c5adf53db41ba2a5aedbcdcb69af5789c045f6bf951e9"} err="failed to get container status \"2ca2f82c83867f1c339c5adf53db41ba2a5aedbcdcb69af5789c045f6bf951e9\": rpc error: code = NotFound desc = could not find container \"2ca2f82c83867f1c339c5adf53db41ba2a5aedbcdcb69af5789c045f6bf951e9\": container with ID starting with 2ca2f82c83867f1c339c5adf53db41ba2a5aedbcdcb69af5789c045f6bf951e9 not found: ID does not exist"
May 28 16:23:14.064562 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.064534    2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-registry-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:23:14.064562 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.064565    2559 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-offline-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:23:14.064733 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.064579    2559 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-ui-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:23:14.064733 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.064591    2559 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/55d96388-8906-435b-827d-d5239d405bdc-online-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:23:14.064733 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.064602    2559 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/55d96388-8906-435b-827d-d5239d405bdc-feast-data\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:23:14.064733 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.064614    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dx6n8\" (UniqueName: \"kubernetes.io/projected/55d96388-8906-435b-827d-d5239d405bdc-kube-api-access-dx6n8\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:23:14.073674 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.073648    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73e02030-e563-4a33-890a-bc6509ff25f3" path="/var/lib/kubelet/pods/73e02030-e563-4a33-890a-bc6509ff25f3/volumes"
May 28 16:23:14.240913 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.240879    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"]
May 28 16:23:14.244352 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:14.244302    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-7rfn2"]
May 28 16:23:16.073995 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:16.073961    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55d96388-8906-435b-827d-d5239d405bdc" path="/var/lib/kubelet/pods/55d96388-8906-435b-827d-d5239d405bdc/volumes"
May 28 16:23:31.761600 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:31.761566    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-jmm99"]
May 28 16:23:31.762057 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:31.761848    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jmm99" podUID="6ed866f3-bb47-4d42-bdc9-c389a0e6072a" containerName="manager" containerID="cri-o://127826ca58d1113ade17b82408f2a94576be728d2a6dd943a47fb4311da5b8ca" gracePeriod=10
May 28 16:23:31.974350 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:31.974302    2559 generic.go:358] "Generic (PLEG): container finished" podID="6ed866f3-bb47-4d42-bdc9-c389a0e6072a" containerID="127826ca58d1113ade17b82408f2a94576be728d2a6dd943a47fb4311da5b8ca" exitCode=0
May 28 16:23:31.974492 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:31.974356    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jmm99" event={"ID":"6ed866f3-bb47-4d42-bdc9-c389a0e6072a","Type":"ContainerDied","Data":"127826ca58d1113ade17b82408f2a94576be728d2a6dd943a47fb4311da5b8ca"}
May 28 16:23:32.002548 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:32.002522    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jmm99"
May 28 16:23:32.187775 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:32.187674    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xcpn\" (UniqueName: \"kubernetes.io/projected/6ed866f3-bb47-4d42-bdc9-c389a0e6072a-kube-api-access-7xcpn\") pod \"6ed866f3-bb47-4d42-bdc9-c389a0e6072a\" (UID: \"6ed866f3-bb47-4d42-bdc9-c389a0e6072a\") "
May 28 16:23:32.189825 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:32.189792    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ed866f3-bb47-4d42-bdc9-c389a0e6072a-kube-api-access-7xcpn" (OuterVolumeSpecName: "kube-api-access-7xcpn") pod "6ed866f3-bb47-4d42-bdc9-c389a0e6072a" (UID: "6ed866f3-bb47-4d42-bdc9-c389a0e6072a"). InnerVolumeSpecName "kube-api-access-7xcpn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 16:23:32.288100 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:32.288071    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xcpn\" (UniqueName: \"kubernetes.io/projected/6ed866f3-bb47-4d42-bdc9-c389a0e6072a-kube-api-access-7xcpn\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:23:32.978296 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:32.978259    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jmm99" event={"ID":"6ed866f3-bb47-4d42-bdc9-c389a0e6072a","Type":"ContainerDied","Data":"e098dfbc9ecd2aeaacf60bd092102ed00f1ea2a5598adcea778dd07468df3310"}
May 28 16:23:32.978296 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:32.978291    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-jmm99"
May 28 16:23:32.978296 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:32.978302    2559 scope.go:117] "RemoveContainer" containerID="127826ca58d1113ade17b82408f2a94576be728d2a6dd943a47fb4311da5b8ca"
May 28 16:23:32.998455 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:32.998427    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-jmm99"]
May 28 16:23:33.001837 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:33.001814    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-jmm99"]
May 28 16:23:34.073288 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:34.073251    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ed866f3-bb47-4d42-bdc9-c389a0e6072a" path="/var/lib/kubelet/pods/6ed866f3-bb47-4d42-bdc9-c389a0e6072a/volumes"
May 28 16:23:36.945527 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945495    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-z5p6z"]
May 28 16:23:36.945920 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945713    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73e02030-e563-4a33-890a-bc6509ff25f3" containerName="offline"
May 28 16:23:36.945920 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945723    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="73e02030-e563-4a33-890a-bc6509ff25f3" containerName="offline"
May 28 16:23:36.945920 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945732    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73e02030-e563-4a33-890a-bc6509ff25f3" containerName="ui"
May 28 16:23:36.945920 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945738    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="73e02030-e563-4a33-890a-bc6509ff25f3" containerName="ui"
May 28 16:23:36.945920 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945746    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="offline"
May 28 16:23:36.945920 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945752    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="offline"
May 28 16:23:36.945920 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945760    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="registry"
May 28 16:23:36.945920 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945765    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="registry"
May 28 16:23:36.945920 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945771    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73e02030-e563-4a33-890a-bc6509ff25f3" containerName="feast-init"
May 28 16:23:36.945920 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945775    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="73e02030-e563-4a33-890a-bc6509ff25f3" containerName="feast-init"
May 28 16:23:36.945920 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945786    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="feast-init"
May 28 16:23:36.945920 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945790    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="feast-init"
May 28 16:23:36.945920 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945798    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="online"
May 28 16:23:36.945920 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945803    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="online"
May 28 16:23:36.945920 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945807    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="ui"
May 28 16:23:36.945920 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945812    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="ui"
May 28 16:23:36.945920 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945818    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ed866f3-bb47-4d42-bdc9-c389a0e6072a" containerName="manager"
May 28 16:23:36.945920 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945823    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ed866f3-bb47-4d42-bdc9-c389a0e6072a" containerName="manager"
May 28 16:23:36.945920 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945828    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73e02030-e563-4a33-890a-bc6509ff25f3" containerName="online"
May 28 16:23:36.945920 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945833    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="73e02030-e563-4a33-890a-bc6509ff25f3" containerName="online"
May 28 16:23:36.945920 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945838    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="feast-apply"
May 28 16:23:36.945920 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945843    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="feast-apply"
May 28 16:23:36.945920 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945849    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73e02030-e563-4a33-890a-bc6509ff25f3" containerName="feast-apply"
May 28 16:23:36.945920 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945853    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="73e02030-e563-4a33-890a-bc6509ff25f3" containerName="feast-apply"
May 28 16:23:36.945920 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945920    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="ui"
May 28 16:23:36.945920 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945929    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="73e02030-e563-4a33-890a-bc6509ff25f3" containerName="ui"
May 28 16:23:36.946644 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945936    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="online"
May 28 16:23:36.946644 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945944    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="6ed866f3-bb47-4d42-bdc9-c389a0e6072a" containerName="manager"
May 28 16:23:36.946644 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945950    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="73e02030-e563-4a33-890a-bc6509ff25f3" containerName="offline"
May 28 16:23:36.946644 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945956    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="registry"
May 28 16:23:36.946644 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945961    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="55d96388-8906-435b-827d-d5239d405bdc" containerName="offline"
May 28 16:23:36.946644 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.945967    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="73e02030-e563-4a33-890a-bc6509ff25f3" containerName="online"
May 28 16:23:36.949993 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.949975    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-z5p6z"
May 28 16:23:36.952637 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.952607    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\""
May 28 16:23:36.953111 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.953087    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\""
May 28 16:23:36.953253 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.953179    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-w2z7v\""
May 28 16:23:36.953424 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:36.953404    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-z5p6z"]
May 28 16:23:37.121419 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:37.121380    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mpdt\" (UniqueName: \"kubernetes.io/projected/1667b6f3-c7d4-4ff3-8374-651b66413ad1-kube-api-access-6mpdt\") pod \"feast-operator-controller-manager-cf99db75-z5p6z\" (UID: \"1667b6f3-c7d4-4ff3-8374-651b66413ad1\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-z5p6z"
May 28 16:23:37.222674 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:37.222600    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6mpdt\" (UniqueName: \"kubernetes.io/projected/1667b6f3-c7d4-4ff3-8374-651b66413ad1-kube-api-access-6mpdt\") pod \"feast-operator-controller-manager-cf99db75-z5p6z\" (UID: \"1667b6f3-c7d4-4ff3-8374-651b66413ad1\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-z5p6z"
May 28 16:23:37.230986 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:37.230960    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mpdt\" (UniqueName: \"kubernetes.io/projected/1667b6f3-c7d4-4ff3-8374-651b66413ad1-kube-api-access-6mpdt\") pod \"feast-operator-controller-manager-cf99db75-z5p6z\" (UID: \"1667b6f3-c7d4-4ff3-8374-651b66413ad1\") " pod="feast-operator-system/feast-operator-controller-manager-cf99db75-z5p6z"
May 28 16:23:37.260882 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:37.260853    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-z5p6z"
May 28 16:23:37.379504 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:37.379472    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-cf99db75-z5p6z"]
May 28 16:23:37.382571 ip-10-0-133-120 kubenswrapper[2559]: W0528 16:23:37.382543    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1667b6f3_c7d4_4ff3_8374_651b66413ad1.slice/crio-faf051641e40760e7576638ba7802a346f3c52e8da06848a5d78757593531d18 WatchSource:0}: Error finding container faf051641e40760e7576638ba7802a346f3c52e8da06848a5d78757593531d18: Status 404 returned error can't find the container with id faf051641e40760e7576638ba7802a346f3c52e8da06848a5d78757593531d18
May 28 16:23:37.384191 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:37.384171    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 16:23:37.994314 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:37.994275    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-z5p6z" event={"ID":"1667b6f3-c7d4-4ff3-8374-651b66413ad1","Type":"ContainerStarted","Data":"782d9df22bc29dac3a72c5eae6538cd5abd33338f67209f1a662d8f8aef203fa"}
May 28 16:23:37.994723 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:37.994317    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-z5p6z" event={"ID":"1667b6f3-c7d4-4ff3-8374-651b66413ad1","Type":"ContainerStarted","Data":"faf051641e40760e7576638ba7802a346f3c52e8da06848a5d78757593531d18"}
May 28 16:23:37.994723 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:37.994384    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-z5p6z"
May 28 16:23:38.008948 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:38.008894    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-z5p6z" podStartSLOduration=2.008875618 podStartE2EDuration="2.008875618s" podCreationTimestamp="2026-05-28 16:23:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:23:38.008572098 +0000 UTC m=+1466.568215609" watchObservedRunningTime="2026-05-28 16:23:38.008875618 +0000 UTC m=+1466.568519117"
May 28 16:23:48.999980 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:48.999951    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-cf99db75-z5p6z"
May 28 16:23:55.284314 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.284276    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"]
May 28 16:23:55.287838 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.287820    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:23:55.290137 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.290107    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 28 16:23:55.290137 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.290120    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 28 16:23:55.290349 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.290317    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 28 16:23:55.290404 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.290379    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 28 16:23:55.290460 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.290427    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 28 16:23:55.290460 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.290443    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-msgg6\""
May 28 16:23:55.290548 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.290446    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 28 16:23:55.296726 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.296694    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"]
May 28 16:23:55.335045 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.335011    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-gkwls\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:23:55.335230 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.335070    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gkwls\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:23:55.335230 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.335091    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gkwls\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:23:55.335230 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.335112    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gkwls\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:23:55.335230 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.335135    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gkwls\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:23:55.335230 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.335152    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fktj4\" (UniqueName: \"kubernetes.io/projected/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-kube-api-access-fktj4\") pod \"feast-simple-feast-setup-664fcfcc58-gkwls\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:23:55.436301 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.436259    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gkwls\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:23:55.436529 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.436316    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gkwls\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:23:55.436529 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.436366    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fktj4\" (UniqueName: \"kubernetes.io/projected/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-kube-api-access-fktj4\") pod \"feast-simple-feast-setup-664fcfcc58-gkwls\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:23:55.436529 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.436408    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-gkwls\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:23:55.436529 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:23:55.436430    2559 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found
May 28 16:23:55.436529 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.436459    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gkwls\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:23:55.436529 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.436486    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gkwls\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:23:55.436529 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:23:55.436489    2559 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found
May 28 16:23:55.436529 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:23:55.436502    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-offline-tls podName:b556f11c-4a92-4ec8-a7f6-2d9ab32e3077 nodeName:}" failed. No retries permitted until 2026-05-28 16:23:55.936484335 +0000 UTC m=+1484.496127811 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-offline-tls") pod "feast-simple-feast-setup-664fcfcc58-gkwls" (UID: "b556f11c-4a92-4ec8-a7f6-2d9ab32e3077") : secret "feast-simple-feast-setup-offline-tls" not found
May 28 16:23:55.436878 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:23:55.436548    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-online-tls podName:b556f11c-4a92-4ec8-a7f6-2d9ab32e3077 nodeName:}" failed. No retries permitted until 2026-05-28 16:23:55.936530479 +0000 UTC m=+1484.496173969 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-online-tls") pod "feast-simple-feast-setup-664fcfcc58-gkwls" (UID: "b556f11c-4a92-4ec8-a7f6-2d9ab32e3077") : secret "feast-simple-feast-setup-online-tls" not found
May 28 16:23:55.436878 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:23:55.436554    2559 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 28 16:23:55.436878 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:23:55.436604    2559 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found
May 28 16:23:55.436878 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:23:55.436611    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-ui-tls podName:b556f11c-4a92-4ec8-a7f6-2d9ab32e3077 nodeName:}" failed. No retries permitted until 2026-05-28 16:23:55.936594319 +0000 UTC m=+1484.496237798 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-ui-tls") pod "feast-simple-feast-setup-664fcfcc58-gkwls" (UID: "b556f11c-4a92-4ec8-a7f6-2d9ab32e3077") : secret "feast-simple-feast-setup-ui-tls" not found
May 28 16:23:55.436878 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:23:55.436680    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-registry-tls podName:b556f11c-4a92-4ec8-a7f6-2d9ab32e3077 nodeName:}" failed. No retries permitted until 2026-05-28 16:23:55.936665179 +0000 UTC m=+1484.496308661 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-registry-tls") pod "feast-simple-feast-setup-664fcfcc58-gkwls" (UID: "b556f11c-4a92-4ec8-a7f6-2d9ab32e3077") : secret "feast-simple-feast-setup-registry-tls" not found
May 28 16:23:55.436878 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.436832    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-gkwls\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:23:55.444929 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.444899    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fktj4\" (UniqueName: \"kubernetes.io/projected/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-kube-api-access-fktj4\") pod \"feast-simple-feast-setup-664fcfcc58-gkwls\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:23:55.940413 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.940377    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gkwls\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:23:55.940413 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.940417    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gkwls\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:23:55.940623 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.940444    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gkwls\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:23:55.940623 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.940473    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gkwls\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:23:55.943048 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.943020    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gkwls\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:23:55.943048 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.943034    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gkwls\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:23:55.943236 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.943133    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gkwls\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:23:55.943236 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:55.943165    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gkwls\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:23:56.197362 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:56.197224    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:23:56.327038 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:56.327002    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"]
May 28 16:23:56.329227 ip-10-0-133-120 kubenswrapper[2559]: W0528 16:23:56.329200    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb556f11c_4a92_4ec8_a7f6_2d9ab32e3077.slice/crio-ac26975a3c59d8fbe523a96ff7786c091c351fbd057b677a1650333166e272b2 WatchSource:0}: Error finding container ac26975a3c59d8fbe523a96ff7786c091c351fbd057b677a1650333166e272b2: Status 404 returned error can't find the container with id ac26975a3c59d8fbe523a96ff7786c091c351fbd057b677a1650333166e272b2
May 28 16:23:57.045976 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:57.045938    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" event={"ID":"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077","Type":"ContainerStarted","Data":"015316504b9faf0cc90c17967d020aec27076a2c5f7ad3f52018d687c8198613"}
May 28 16:23:57.045976 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:23:57.045978    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" event={"ID":"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077","Type":"ContainerStarted","Data":"ac26975a3c59d8fbe523a96ff7786c091c351fbd057b677a1650333166e272b2"}
May 28 16:24:01.057296 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:01.057262    2559 generic.go:358] "Generic (PLEG): container finished" podID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerID="015316504b9faf0cc90c17967d020aec27076a2c5f7ad3f52018d687c8198613" exitCode=0
May 28 16:24:01.057724 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:01.057303    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" event={"ID":"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077","Type":"ContainerDied","Data":"015316504b9faf0cc90c17967d020aec27076a2c5f7ad3f52018d687c8198613"}
May 28 16:24:02.061076 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:02.061045    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" event={"ID":"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077","Type":"ContainerStarted","Data":"07f38a6e090fe7f6a153825b5338514bda91807c29f55d62afa52b1366c442c3"}
May 28 16:24:07.076568 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:07.076531    2559 generic.go:358] "Generic (PLEG): container finished" podID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerID="07f38a6e090fe7f6a153825b5338514bda91807c29f55d62afa52b1366c442c3" exitCode=0
May 28 16:24:07.076951 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:07.076574    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" event={"ID":"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077","Type":"ContainerDied","Data":"07f38a6e090fe7f6a153825b5338514bda91807c29f55d62afa52b1366c442c3"}
May 28 16:24:08.083281 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:08.083231    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" event={"ID":"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077","Type":"ContainerStarted","Data":"18826088b490a5130c0c3cc80b7aab70252bc1af1ff979b428fbc98d4f8631d4"}
May 28 16:24:08.083788 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:08.083290    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" event={"ID":"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077","Type":"ContainerStarted","Data":"ee6b759eaacc7f491370369839749232f9f7c8a0ba299702ba032818128d5100"}
May 28 16:24:08.083788 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:08.083305    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" event={"ID":"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077","Type":"ContainerStarted","Data":"a7a20a37dd551ea3cf1a31a7ced8142084cd0c0e15467a544f6a350abfe6c79c"}
May 28 16:24:08.083788 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:08.083317    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" event={"ID":"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077","Type":"ContainerStarted","Data":"418200c8903f42fdcf8a00da7b2708a95f1190d1fc4eed22ddb872d6521cfc39"}
May 28 16:24:08.108437 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:08.108357    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" podStartSLOduration=13.108318345 podStartE2EDuration="13.108318345s" podCreationTimestamp="2026-05-28 16:23:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:24:08.106645065 +0000 UTC m=+1496.666288566" watchObservedRunningTime="2026-05-28 16:24:08.108318345 +0000 UTC m=+1496.667961844"
May 28 16:24:08.198204 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:08.198161    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:24:08.198428 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:08.198220    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:24:08.198428 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:08.198235    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:24:08.198428 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:08.198247    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:24:08.200248 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:08.200195    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.19:8443: connect: connection refused"
May 28 16:24:08.200398 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:08.200274    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.19:6571: connect: connection refused"
May 28 16:24:08.200398 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:08.200305    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="online" probeResult="failure" output="Get \"https://10.132.0.19:6567/health\": dial tcp 10.132.0.19:6567: connect: connection refused"
May 28 16:24:08.200398 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:08.200386    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.19:8816: connect: connection refused"
May 28 16:24:11.198682 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:11.198637    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.19:6571: connect: connection refused"
May 28 16:24:11.199264 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:11.198649    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.19:8816: connect: connection refused"
May 28 16:24:11.199417 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:11.198660    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.19:8443: connect: connection refused"
May 28 16:24:11.199417 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:11.198709    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="online" probeResult="failure" output="Get \"https://10.132.0.19:6567/health\": dial tcp 10.132.0.19:6567: connect: connection refused"
May 28 16:24:14.199065 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:14.199031    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:24:14.199542 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:14.199180    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:24:14.199542 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:14.199491    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:24:14.199841 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:14.199815    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:24:14.200055 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:14.200040    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:24:14.200152 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:14.200087    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:24:14.200200 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:14.200152    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:24:14.200297 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:14.200283    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:24:14.200375 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:14.200354    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:24:14.203655 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:14.203632    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:24:15.104209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:15.104176    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:24:15.107068 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:24:15.107043    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:25:16.435126 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:16.435090    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"]
May 28 16:25:16.435648 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:16.435475    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="registry" containerID="cri-o://418200c8903f42fdcf8a00da7b2708a95f1190d1fc4eed22ddb872d6521cfc39" gracePeriod=30
May 28 16:25:16.435648 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:16.435501    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="online" containerID="cri-o://a7a20a37dd551ea3cf1a31a7ced8142084cd0c0e15467a544f6a350abfe6c79c" gracePeriod=30
May 28 16:25:16.435648 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:16.435525    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="offline" containerID="cri-o://ee6b759eaacc7f491370369839749232f9f7c8a0ba299702ba032818128d5100" gracePeriod=30
May 28 16:25:16.435648 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:16.435645    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="ui" containerID="cri-o://18826088b490a5130c0c3cc80b7aab70252bc1af1ff979b428fbc98d4f8631d4" gracePeriod=30
May 28 16:25:16.837877 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:16.837835    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"]
May 28 16:25:16.841315 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:16.841289    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:16.843636 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:16.843608    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-h7gl6\""
May 28 16:25:16.853432 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:16.853405    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"]
May 28 16:25:16.922096 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:16.922056    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmvt6\" (UniqueName: \"kubernetes.io/projected/0855f896-0bf9-4db8-8348-b0cf983ddb3f-kube-api-access-vmvt6\") pod \"feast-simple-feast-setup-664fcfcc58-t5pdk\" (UID: \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:16.922096 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:16.922101    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-t5pdk\" (UID: \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:16.922557 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:16.922123    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0855f896-0bf9-4db8-8348-b0cf983ddb3f-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-t5pdk\" (UID: \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:16.922557 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:16.922219    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-t5pdk\" (UID: \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:16.922557 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:16.922256    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-t5pdk\" (UID: \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:16.922557 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:16.922340    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-t5pdk\" (UID: \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:17.023647 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:17.023607    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0855f896-0bf9-4db8-8348-b0cf983ddb3f-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-t5pdk\" (UID: \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:17.023840 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:17.023663    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-t5pdk\" (UID: \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:17.023840 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:17.023679    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-t5pdk\" (UID: \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:17.023840 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:17.023703    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-t5pdk\" (UID: \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:17.023840 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:17.023743    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vmvt6\" (UniqueName: \"kubernetes.io/projected/0855f896-0bf9-4db8-8348-b0cf983ddb3f-kube-api-access-vmvt6\") pod \"feast-simple-feast-setup-664fcfcc58-t5pdk\" (UID: \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:17.023840 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:17.023781    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-t5pdk\" (UID: \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:17.023840 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:25:17.023797    2559 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found
May 28 16:25:17.024169 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:25:17.023866    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-offline-tls podName:0855f896-0bf9-4db8-8348-b0cf983ddb3f nodeName:}" failed. No retries permitted until 2026-05-28 16:25:17.523844966 +0000 UTC m=+1566.083488459 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-offline-tls") pod "feast-simple-feast-setup-664fcfcc58-t5pdk" (UID: "0855f896-0bf9-4db8-8348-b0cf983ddb3f") : secret "feast-simple-feast-setup-offline-tls" not found
May 28 16:25:17.024169 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:17.024086    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0855f896-0bf9-4db8-8348-b0cf983ddb3f-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-t5pdk\" (UID: \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:17.026236 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:17.026209    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-t5pdk\" (UID: \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:17.026389 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:17.026366    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-t5pdk\" (UID: \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:17.026436 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:17.026407    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-t5pdk\" (UID: \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:17.040364 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:17.035795    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmvt6\" (UniqueName: \"kubernetes.io/projected/0855f896-0bf9-4db8-8348-b0cf983ddb3f-kube-api-access-vmvt6\") pod \"feast-simple-feast-setup-664fcfcc58-t5pdk\" (UID: \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:17.267536 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:17.267502    2559 generic.go:358] "Generic (PLEG): container finished" podID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerID="18826088b490a5130c0c3cc80b7aab70252bc1af1ff979b428fbc98d4f8631d4" exitCode=0
May 28 16:25:17.267536 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:17.267529    2559 generic.go:358] "Generic (PLEG): container finished" podID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerID="a7a20a37dd551ea3cf1a31a7ced8142084cd0c0e15467a544f6a350abfe6c79c" exitCode=0
May 28 16:25:17.267741 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:17.267561    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" event={"ID":"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077","Type":"ContainerDied","Data":"18826088b490a5130c0c3cc80b7aab70252bc1af1ff979b428fbc98d4f8631d4"}
May 28 16:25:17.267741 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:17.267583    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" event={"ID":"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077","Type":"ContainerDied","Data":"a7a20a37dd551ea3cf1a31a7ced8142084cd0c0e15467a544f6a350abfe6c79c"}
May 28 16:25:17.527710 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:17.527661    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-t5pdk\" (UID: \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:17.530143 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:17.530123    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-t5pdk\" (UID: \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:17.750587 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:17.750551    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:17.881997 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:17.881948    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"]
May 28 16:25:17.886542 ip-10-0-133-120 kubenswrapper[2559]: W0528 16:25:17.886515    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0855f896_0bf9_4db8_8348_b0cf983ddb3f.slice/crio-15971ca515d07dc59dca0838609db8e0387bd97005c541af6153d050563d9bc8 WatchSource:0}: Error finding container 15971ca515d07dc59dca0838609db8e0387bd97005c541af6153d050563d9bc8: Status 404 returned error can't find the container with id 15971ca515d07dc59dca0838609db8e0387bd97005c541af6153d050563d9bc8
May 28 16:25:18.271754 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:18.271721    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" event={"ID":"0855f896-0bf9-4db8-8348-b0cf983ddb3f","Type":"ContainerStarted","Data":"1d47c0a3e9c78e990e1bbdf81df57b260570fa97e50abaa4ea72aa29cba58172"}
May 28 16:25:18.271754 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:18.271757    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" event={"ID":"0855f896-0bf9-4db8-8348-b0cf983ddb3f","Type":"ContainerStarted","Data":"15971ca515d07dc59dca0838609db8e0387bd97005c541af6153d050563d9bc8"}
May 28 16:25:23.286525 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:23.286489    2559 generic.go:358] "Generic (PLEG): container finished" podID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerID="1d47c0a3e9c78e990e1bbdf81df57b260570fa97e50abaa4ea72aa29cba58172" exitCode=0
May 28 16:25:23.286929 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:23.286552    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" event={"ID":"0855f896-0bf9-4db8-8348-b0cf983ddb3f","Type":"ContainerDied","Data":"1d47c0a3e9c78e990e1bbdf81df57b260570fa97e50abaa4ea72aa29cba58172"}
May 28 16:25:24.200273 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:24.200225    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.19:8443: connect: connection refused"
May 28 16:25:24.295347 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:24.295296    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" event={"ID":"0855f896-0bf9-4db8-8348-b0cf983ddb3f","Type":"ContainerStarted","Data":"5fabe2a303bd1f562b66e1707361aa8a3fa3088f489dc0ab9f78ab7c197360cd"}
May 28 16:25:25.104472 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:25.104433    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="online" probeResult="failure" output="Get \"https://10.132.0.19:6567/health\": dial tcp 10.132.0.19:6567: connect: connection refused"
May 28 16:25:29.311277 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:29.311246    2559 generic.go:358] "Generic (PLEG): container finished" podID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerID="5fabe2a303bd1f562b66e1707361aa8a3fa3088f489dc0ab9f78ab7c197360cd" exitCode=0
May 28 16:25:29.311699 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:29.311322    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" event={"ID":"0855f896-0bf9-4db8-8348-b0cf983ddb3f","Type":"ContainerDied","Data":"5fabe2a303bd1f562b66e1707361aa8a3fa3088f489dc0ab9f78ab7c197360cd"}
May 28 16:25:30.318575 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:30.318530    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" event={"ID":"0855f896-0bf9-4db8-8348-b0cf983ddb3f","Type":"ContainerStarted","Data":"658a5d76605edbf76e63cc1a1b98bf04dacac956f32acf308b7908ee61bb7a8e"}
May 28 16:25:30.318575 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:30.318576    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" event={"ID":"0855f896-0bf9-4db8-8348-b0cf983ddb3f","Type":"ContainerStarted","Data":"9e3f2b322e89634a90cc56e813adcb2461833323d00ef373420b0303730177f0"}
May 28 16:25:30.319128 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:30.318592    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" event={"ID":"0855f896-0bf9-4db8-8348-b0cf983ddb3f","Type":"ContainerStarted","Data":"90edb2fbec7b87e97a9149749f4dfda0034428af4d47d24bd80f618531aae12c"}
May 28 16:25:30.319128 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:30.318605    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" event={"ID":"0855f896-0bf9-4db8-8348-b0cf983ddb3f","Type":"ContainerStarted","Data":"7b1f6cf77e608d1c7061f907bf86eb133a002ca6a261436bbec0ec9744928dbe"}
May 28 16:25:30.343195 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:30.343132    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" podStartSLOduration=14.343114265 podStartE2EDuration="14.343114265s" podCreationTimestamp="2026-05-28 16:25:16 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:25:30.341923979 +0000 UTC m=+1578.901567479" watchObservedRunningTime="2026-05-28 16:25:30.343114265 +0000 UTC m=+1578.902757764"
May 28 16:25:32.751526 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:32.751475    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:32.751526 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:32.751535    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:32.752160 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:32.752129    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:32.752367 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:32.752346    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:32.753717 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:32.753682    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.20:8443: connect: connection refused"
May 28 16:25:32.753829 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:32.753714    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="online" probeResult="failure" output="Get \"https://10.132.0.20:6567/health\": dial tcp 10.132.0.20:6567: connect: connection refused"
May 28 16:25:32.753829 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:32.753817    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.20:8816: connect: connection refused"
May 28 16:25:32.753947 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:32.753842    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.20:6571: connect: connection refused"
May 28 16:25:34.200989 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:34.200940    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.19:8443: connect: connection refused"
May 28 16:25:35.105089 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:35.105041    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="online" probeResult="failure" output="Get \"https://10.132.0.19:6567/health\": dial tcp 10.132.0.19:6567: connect: connection refused"
May 28 16:25:35.752618 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:35.752586    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:35.753083 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:35.752694    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:35.753083 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:35.753011    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:35.753406 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:35.753382    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:35.753560 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:35.753416    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:35.753560 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:35.753430    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:35.753841 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:35.753822    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:35.753914 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:35.753886    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:35.753946 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:35.753922    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:35.757137 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:35.757122    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:36.341370 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:36.341336    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:36.344286 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:36.344263    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:25:44.200078 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:44.200029    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.19:8443: connect: connection refused"
May 28 16:25:44.200594 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:44.200169    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:25:45.105231 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:45.105185    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="online" probeResult="failure" output="Get \"https://10.132.0.19:6567/health\": dial tcp 10.132.0.19:6567: connect: connection refused"
May 28 16:25:45.105443 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:45.105358    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:25:47.082547 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.082524    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:25:47.158173 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.158136    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-online-tls\") pod \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") "
May 28 16:25:47.158340 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.158184    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-offline-tls\") pod \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") "
May 28 16:25:47.158340 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.158208    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-registry-tls\") pod \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") "
May 28 16:25:47.158340 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.158231    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fktj4\" (UniqueName: \"kubernetes.io/projected/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-kube-api-access-fktj4\") pod \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") "
May 28 16:25:47.158523 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.158382    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-feast-data\") pod \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") "
May 28 16:25:47.158523 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.158444    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-ui-tls\") pod \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\" (UID: \"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077\") "
May 28 16:25:47.158972 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.158927    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-feast-data" (OuterVolumeSpecName: "feast-data") pod "b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" (UID: "b556f11c-4a92-4ec8-a7f6-2d9ab32e3077"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 16:25:47.160622 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.160599    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-online-tls" (OuterVolumeSpecName: "online-tls") pod "b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" (UID: "b556f11c-4a92-4ec8-a7f6-2d9ab32e3077"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:25:47.160751 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.160624    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" (UID: "b556f11c-4a92-4ec8-a7f6-2d9ab32e3077"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:25:47.160751 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.160650    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" (UID: "b556f11c-4a92-4ec8-a7f6-2d9ab32e3077"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:25:47.160872 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.160749    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" (UID: "b556f11c-4a92-4ec8-a7f6-2d9ab32e3077"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:25:47.160872 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.160765    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-kube-api-access-fktj4" (OuterVolumeSpecName: "kube-api-access-fktj4") pod "b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" (UID: "b556f11c-4a92-4ec8-a7f6-2d9ab32e3077"). InnerVolumeSpecName "kube-api-access-fktj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 16:25:47.259294 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.259250    2559 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-feast-data\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:25:47.259294 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.259289    2559 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-ui-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:25:47.259294 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.259298    2559 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-online-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:25:47.259541 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.259309    2559 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-offline-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:25:47.259541 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.259317    2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-registry-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:25:47.259541 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.259350    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fktj4\" (UniqueName: \"kubernetes.io/projected/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077-kube-api-access-fktj4\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:25:47.373197 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.373163    2559 generic.go:358] "Generic (PLEG): container finished" podID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerID="ee6b759eaacc7f491370369839749232f9f7c8a0ba299702ba032818128d5100" exitCode=137
May 28 16:25:47.373197 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.373188    2559 generic.go:358] "Generic (PLEG): container finished" podID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerID="418200c8903f42fdcf8a00da7b2708a95f1190d1fc4eed22ddb872d6521cfc39" exitCode=137
May 28 16:25:47.373449 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.373237    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"
May 28 16:25:47.373449 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.373244    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" event={"ID":"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077","Type":"ContainerDied","Data":"ee6b759eaacc7f491370369839749232f9f7c8a0ba299702ba032818128d5100"}
May 28 16:25:47.373449 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.373291    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" event={"ID":"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077","Type":"ContainerDied","Data":"418200c8903f42fdcf8a00da7b2708a95f1190d1fc4eed22ddb872d6521cfc39"}
May 28 16:25:47.373449 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.373306    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls" event={"ID":"b556f11c-4a92-4ec8-a7f6-2d9ab32e3077","Type":"ContainerDied","Data":"ac26975a3c59d8fbe523a96ff7786c091c351fbd057b677a1650333166e272b2"}
May 28 16:25:47.373449 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.373344    2559 scope.go:117] "RemoveContainer" containerID="18826088b490a5130c0c3cc80b7aab70252bc1af1ff979b428fbc98d4f8631d4"
May 28 16:25:47.381668 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.381654    2559 scope.go:117] "RemoveContainer" containerID="ee6b759eaacc7f491370369839749232f9f7c8a0ba299702ba032818128d5100"
May 28 16:25:47.388316 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.388300    2559 scope.go:117] "RemoveContainer" containerID="a7a20a37dd551ea3cf1a31a7ced8142084cd0c0e15467a544f6a350abfe6c79c"
May 28 16:25:47.395282 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.395232    2559 scope.go:117] "RemoveContainer" containerID="418200c8903f42fdcf8a00da7b2708a95f1190d1fc4eed22ddb872d6521cfc39"
May 28 16:25:47.395702 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.395682    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"]
May 28 16:25:47.397869 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.397845    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-gkwls"]
May 28 16:25:47.402666 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.402646    2559 scope.go:117] "RemoveContainer" containerID="07f38a6e090fe7f6a153825b5338514bda91807c29f55d62afa52b1366c442c3"
May 28 16:25:47.409182 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.409166    2559 scope.go:117] "RemoveContainer" containerID="015316504b9faf0cc90c17967d020aec27076a2c5f7ad3f52018d687c8198613"
May 28 16:25:47.415561 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.415543    2559 scope.go:117] "RemoveContainer" containerID="18826088b490a5130c0c3cc80b7aab70252bc1af1ff979b428fbc98d4f8631d4"
May 28 16:25:47.415816 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:25:47.415799    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18826088b490a5130c0c3cc80b7aab70252bc1af1ff979b428fbc98d4f8631d4\": container with ID starting with 18826088b490a5130c0c3cc80b7aab70252bc1af1ff979b428fbc98d4f8631d4 not found: ID does not exist" containerID="18826088b490a5130c0c3cc80b7aab70252bc1af1ff979b428fbc98d4f8631d4"
May 28 16:25:47.415860 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.415828    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18826088b490a5130c0c3cc80b7aab70252bc1af1ff979b428fbc98d4f8631d4"} err="failed to get container status \"18826088b490a5130c0c3cc80b7aab70252bc1af1ff979b428fbc98d4f8631d4\": rpc error: code = NotFound desc = could not find container \"18826088b490a5130c0c3cc80b7aab70252bc1af1ff979b428fbc98d4f8631d4\": container with ID starting with 18826088b490a5130c0c3cc80b7aab70252bc1af1ff979b428fbc98d4f8631d4 not found: ID does not exist"
May 28 16:25:47.415860 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.415848    2559 scope.go:117] "RemoveContainer" containerID="ee6b759eaacc7f491370369839749232f9f7c8a0ba299702ba032818128d5100"
May 28 16:25:47.416049 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:25:47.416034    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee6b759eaacc7f491370369839749232f9f7c8a0ba299702ba032818128d5100\": container with ID starting with ee6b759eaacc7f491370369839749232f9f7c8a0ba299702ba032818128d5100 not found: ID does not exist" containerID="ee6b759eaacc7f491370369839749232f9f7c8a0ba299702ba032818128d5100"
May 28 16:25:47.416088 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.416054    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee6b759eaacc7f491370369839749232f9f7c8a0ba299702ba032818128d5100"} err="failed to get container status \"ee6b759eaacc7f491370369839749232f9f7c8a0ba299702ba032818128d5100\": rpc error: code = NotFound desc = could not find container \"ee6b759eaacc7f491370369839749232f9f7c8a0ba299702ba032818128d5100\": container with ID starting with ee6b759eaacc7f491370369839749232f9f7c8a0ba299702ba032818128d5100 not found: ID does not exist"
May 28 16:25:47.416088 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.416068    2559 scope.go:117] "RemoveContainer" containerID="a7a20a37dd551ea3cf1a31a7ced8142084cd0c0e15467a544f6a350abfe6c79c"
May 28 16:25:47.416240 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:25:47.416223    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7a20a37dd551ea3cf1a31a7ced8142084cd0c0e15467a544f6a350abfe6c79c\": container with ID starting with a7a20a37dd551ea3cf1a31a7ced8142084cd0c0e15467a544f6a350abfe6c79c not found: ID does not exist" containerID="a7a20a37dd551ea3cf1a31a7ced8142084cd0c0e15467a544f6a350abfe6c79c"
May 28 16:25:47.416278 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.416243    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7a20a37dd551ea3cf1a31a7ced8142084cd0c0e15467a544f6a350abfe6c79c"} err="failed to get container status \"a7a20a37dd551ea3cf1a31a7ced8142084cd0c0e15467a544f6a350abfe6c79c\": rpc error: code = NotFound desc = could not find container \"a7a20a37dd551ea3cf1a31a7ced8142084cd0c0e15467a544f6a350abfe6c79c\": container with ID starting with a7a20a37dd551ea3cf1a31a7ced8142084cd0c0e15467a544f6a350abfe6c79c not found: ID does not exist"
May 28 16:25:47.416278 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.416258    2559 scope.go:117] "RemoveContainer" containerID="418200c8903f42fdcf8a00da7b2708a95f1190d1fc4eed22ddb872d6521cfc39"
May 28 16:25:47.416489 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:25:47.416472    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"418200c8903f42fdcf8a00da7b2708a95f1190d1fc4eed22ddb872d6521cfc39\": container with ID starting with 418200c8903f42fdcf8a00da7b2708a95f1190d1fc4eed22ddb872d6521cfc39 not found: ID does not exist" containerID="418200c8903f42fdcf8a00da7b2708a95f1190d1fc4eed22ddb872d6521cfc39"
May 28 16:25:47.416539 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.416496    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"418200c8903f42fdcf8a00da7b2708a95f1190d1fc4eed22ddb872d6521cfc39"} err="failed to get container status \"418200c8903f42fdcf8a00da7b2708a95f1190d1fc4eed22ddb872d6521cfc39\": rpc error: code = NotFound desc = could not find container \"418200c8903f42fdcf8a00da7b2708a95f1190d1fc4eed22ddb872d6521cfc39\": container with ID starting with 418200c8903f42fdcf8a00da7b2708a95f1190d1fc4eed22ddb872d6521cfc39 not found: ID does not exist"
May 28 16:25:47.416539 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.416509    2559 scope.go:117] "RemoveContainer" containerID="07f38a6e090fe7f6a153825b5338514bda91807c29f55d62afa52b1366c442c3"
May 28 16:25:47.416770 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:25:47.416747    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07f38a6e090fe7f6a153825b5338514bda91807c29f55d62afa52b1366c442c3\": container with ID starting with 07f38a6e090fe7f6a153825b5338514bda91807c29f55d62afa52b1366c442c3 not found: ID does not exist" containerID="07f38a6e090fe7f6a153825b5338514bda91807c29f55d62afa52b1366c442c3"
May 28 16:25:47.416859 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.416779    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07f38a6e090fe7f6a153825b5338514bda91807c29f55d62afa52b1366c442c3"} err="failed to get container status \"07f38a6e090fe7f6a153825b5338514bda91807c29f55d62afa52b1366c442c3\": rpc error: code = NotFound desc = could not find container \"07f38a6e090fe7f6a153825b5338514bda91807c29f55d62afa52b1366c442c3\": container with ID starting with 07f38a6e090fe7f6a153825b5338514bda91807c29f55d62afa52b1366c442c3 not found: ID does not exist"
May 28 16:25:47.416859 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.416802    2559 scope.go:117] "RemoveContainer" containerID="015316504b9faf0cc90c17967d020aec27076a2c5f7ad3f52018d687c8198613"
May 28 16:25:47.417050 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:25:47.417033    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"015316504b9faf0cc90c17967d020aec27076a2c5f7ad3f52018d687c8198613\": container with ID starting with 015316504b9faf0cc90c17967d020aec27076a2c5f7ad3f52018d687c8198613 not found: ID does not exist" containerID="015316504b9faf0cc90c17967d020aec27076a2c5f7ad3f52018d687c8198613"
May 28 16:25:47.417090 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.417056    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"015316504b9faf0cc90c17967d020aec27076a2c5f7ad3f52018d687c8198613"} err="failed to get container status \"015316504b9faf0cc90c17967d020aec27076a2c5f7ad3f52018d687c8198613\": rpc error: code = NotFound desc = could not find container \"015316504b9faf0cc90c17967d020aec27076a2c5f7ad3f52018d687c8198613\": container with ID starting with 015316504b9faf0cc90c17967d020aec27076a2c5f7ad3f52018d687c8198613 not found: ID does not exist"
May 28 16:25:47.417090 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.417070    2559 scope.go:117] "RemoveContainer" containerID="18826088b490a5130c0c3cc80b7aab70252bc1af1ff979b428fbc98d4f8631d4"
May 28 16:25:47.417262 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.417246    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18826088b490a5130c0c3cc80b7aab70252bc1af1ff979b428fbc98d4f8631d4"} err="failed to get container status \"18826088b490a5130c0c3cc80b7aab70252bc1af1ff979b428fbc98d4f8631d4\": rpc error: code = NotFound desc = could not find container \"18826088b490a5130c0c3cc80b7aab70252bc1af1ff979b428fbc98d4f8631d4\": container with ID starting with 18826088b490a5130c0c3cc80b7aab70252bc1af1ff979b428fbc98d4f8631d4 not found: ID does not exist"
May 28 16:25:47.417315 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.417263    2559 scope.go:117] "RemoveContainer" containerID="ee6b759eaacc7f491370369839749232f9f7c8a0ba299702ba032818128d5100"
May 28 16:25:47.417518 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.417499    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee6b759eaacc7f491370369839749232f9f7c8a0ba299702ba032818128d5100"} err="failed to get container status \"ee6b759eaacc7f491370369839749232f9f7c8a0ba299702ba032818128d5100\": rpc error: code = NotFound desc = could not find container \"ee6b759eaacc7f491370369839749232f9f7c8a0ba299702ba032818128d5100\": container with ID starting with ee6b759eaacc7f491370369839749232f9f7c8a0ba299702ba032818128d5100 not found: ID does not exist"
May 28 16:25:47.417570 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.417518    2559 scope.go:117] "RemoveContainer" containerID="a7a20a37dd551ea3cf1a31a7ced8142084cd0c0e15467a544f6a350abfe6c79c"
May 28 16:25:47.417743 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.417724    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7a20a37dd551ea3cf1a31a7ced8142084cd0c0e15467a544f6a350abfe6c79c"} err="failed to get container status \"a7a20a37dd551ea3cf1a31a7ced8142084cd0c0e15467a544f6a350abfe6c79c\": rpc error: code = NotFound desc = could not find container \"a7a20a37dd551ea3cf1a31a7ced8142084cd0c0e15467a544f6a350abfe6c79c\": container with ID starting with a7a20a37dd551ea3cf1a31a7ced8142084cd0c0e15467a544f6a350abfe6c79c not found: ID does not exist"
May 28 16:25:47.417812 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.417745    2559 scope.go:117] "RemoveContainer" containerID="418200c8903f42fdcf8a00da7b2708a95f1190d1fc4eed22ddb872d6521cfc39"
May 28 16:25:47.417949 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.417933    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"418200c8903f42fdcf8a00da7b2708a95f1190d1fc4eed22ddb872d6521cfc39"} err="failed to get container status \"418200c8903f42fdcf8a00da7b2708a95f1190d1fc4eed22ddb872d6521cfc39\": rpc error: code = NotFound desc = could not find container \"418200c8903f42fdcf8a00da7b2708a95f1190d1fc4eed22ddb872d6521cfc39\": container with ID starting with 418200c8903f42fdcf8a00da7b2708a95f1190d1fc4eed22ddb872d6521cfc39 not found: ID does not exist"
May 28 16:25:47.418004 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.417949    2559 scope.go:117] "RemoveContainer" containerID="07f38a6e090fe7f6a153825b5338514bda91807c29f55d62afa52b1366c442c3"
May 28 16:25:47.418103 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.418090    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07f38a6e090fe7f6a153825b5338514bda91807c29f55d62afa52b1366c442c3"} err="failed to get container status \"07f38a6e090fe7f6a153825b5338514bda91807c29f55d62afa52b1366c442c3\": rpc error: code = NotFound desc = could not find container \"07f38a6e090fe7f6a153825b5338514bda91807c29f55d62afa52b1366c442c3\": container with ID starting with 07f38a6e090fe7f6a153825b5338514bda91807c29f55d62afa52b1366c442c3 not found: ID does not exist"
May 28 16:25:47.418103 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.418103    2559 scope.go:117] "RemoveContainer" containerID="015316504b9faf0cc90c17967d020aec27076a2c5f7ad3f52018d687c8198613"
May 28 16:25:47.418311 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:47.418295    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"015316504b9faf0cc90c17967d020aec27076a2c5f7ad3f52018d687c8198613"} err="failed to get container status \"015316504b9faf0cc90c17967d020aec27076a2c5f7ad3f52018d687c8198613\": rpc error: code = NotFound desc = could not find container \"015316504b9faf0cc90c17967d020aec27076a2c5f7ad3f52018d687c8198613\": container with ID starting with 015316504b9faf0cc90c17967d020aec27076a2c5f7ad3f52018d687c8198613 not found: ID does not exist"
May 28 16:25:48.073901 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:25:48.073870    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" path="/var/lib/kubelet/pods/b556f11c-4a92-4ec8-a7f6-2d9ab32e3077/volumes"
May 28 16:26:38.368450 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.368414    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"]
May 28 16:26:38.369067 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.368641    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="ui"
May 28 16:26:38.369067 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.368654    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="ui"
May 28 16:26:38.369067 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.368671    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="feast-init"
May 28 16:26:38.369067 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.368678    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="feast-init"
May 28 16:26:38.369067 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.368685    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="online"
May 28 16:26:38.369067 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.368692    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="online"
May 28 16:26:38.369067 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.368698    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="offline"
May 28 16:26:38.369067 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.368703    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="offline"
May 28 16:26:38.369067 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.368716    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="feast-apply"
May 28 16:26:38.369067 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.368730    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="feast-apply"
May 28 16:26:38.369067 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.368737    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="registry"
May 28 16:26:38.369067 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.368742    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="registry"
May 28 16:26:38.369067 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.368778    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="offline"
May 28 16:26:38.369067 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.368786    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="ui"
May 28 16:26:38.369067 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.368792    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="online"
May 28 16:26:38.369067 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.368799    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="b556f11c-4a92-4ec8-a7f6-2d9ab32e3077" containerName="registry"
May 28 16:26:38.376199 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.376174    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:38.378681 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.378650    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-fdxh7\""
May 28 16:26:38.378681 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.378652    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 28 16:26:38.378879 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.378862    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 28 16:26:38.378939 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.378903    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 28 16:26:38.378939 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.378918    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 28 16:26:38.379041 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.378943    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 28 16:26:38.379041 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.378953    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 28 16:26:38.383467 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.383016    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"]
May 28 16:26:38.437009 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.436968    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-njgr4\" (UID: \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:38.437009 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.437013    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-njgr4\" (UID: \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:38.437380 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.437039    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-njgr4\" (UID: \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:38.437380 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.437111    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-njgr4\" (UID: \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:38.437380 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.437147    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-njgr4\" (UID: \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:38.437380 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.437165    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2mf6\" (UniqueName: \"kubernetes.io/projected/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-kube-api-access-s2mf6\") pod \"feast-simple-feast-remote-setup-5cff94746b-njgr4\" (UID: \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:38.538152 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.538122    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-njgr4\" (UID: \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:38.538313 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.538162    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-njgr4\" (UID: \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:38.538313 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.538186    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-njgr4\" (UID: \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:38.538313 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.538226    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-njgr4\" (UID: \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:38.538313 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.538244    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-njgr4\" (UID: \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:38.538313 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.538263    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s2mf6\" (UniqueName: \"kubernetes.io/projected/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-kube-api-access-s2mf6\") pod \"feast-simple-feast-remote-setup-5cff94746b-njgr4\" (UID: \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:38.538539 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:26:38.538398    2559 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found
May 28 16:26:38.538539 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:26:38.538498    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-offline-tls podName:9b4fcbba-c7d2-459b-a8a9-243fee5d691e nodeName:}" failed. No retries permitted until 2026-05-28 16:26:39.03847223 +0000 UTC m=+1647.598115706 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-offline-tls") pod "feast-simple-feast-remote-setup-5cff94746b-njgr4" (UID: "9b4fcbba-c7d2-459b-a8a9-243fee5d691e") : secret "feast-simple-feast-remote-setup-offline-tls" not found
May 28 16:26:38.538653 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.538594    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-njgr4\" (UID: \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:38.538944 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.538917    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-njgr4\" (UID: \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:38.540606 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.540579    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-njgr4\" (UID: \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:38.540711 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.540670    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-njgr4\" (UID: \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:38.548208 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:38.548188    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2mf6\" (UniqueName: \"kubernetes.io/projected/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-kube-api-access-s2mf6\") pod \"feast-simple-feast-remote-setup-5cff94746b-njgr4\" (UID: \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:39.042476 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:39.042425    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-njgr4\" (UID: \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:39.044794 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:39.044772    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-njgr4\" (UID: \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:39.286449 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:39.286420    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:39.420357 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:39.420310    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"]
May 28 16:26:39.422191 ip-10-0-133-120 kubenswrapper[2559]: W0528 16:26:39.422166    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b4fcbba_c7d2_459b_a8a9_243fee5d691e.slice/crio-891c8394820bc30e2d0477c7ee7e5602338bdc9a1a163e512c9f9c1812fc0a79 WatchSource:0}: Error finding container 891c8394820bc30e2d0477c7ee7e5602338bdc9a1a163e512c9f9c1812fc0a79: Status 404 returned error can't find the container with id 891c8394820bc30e2d0477c7ee7e5602338bdc9a1a163e512c9f9c1812fc0a79
May 28 16:26:39.515101 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:39.515066    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4" event={"ID":"9b4fcbba-c7d2-459b-a8a9-243fee5d691e","Type":"ContainerStarted","Data":"e4c5b9081e123dd641ca70034d1eca0a6b238334082ab481b6e34e2bad931817"}
May 28 16:26:39.515101 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:39.515107    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4" event={"ID":"9b4fcbba-c7d2-459b-a8a9-243fee5d691e","Type":"ContainerStarted","Data":"891c8394820bc30e2d0477c7ee7e5602338bdc9a1a163e512c9f9c1812fc0a79"}
May 28 16:26:44.529476 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:44.529442    2559 generic.go:358] "Generic (PLEG): container finished" podID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerID="e4c5b9081e123dd641ca70034d1eca0a6b238334082ab481b6e34e2bad931817" exitCode=0
May 28 16:26:44.529865 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:44.529483    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4" event={"ID":"9b4fcbba-c7d2-459b-a8a9-243fee5d691e","Type":"ContainerDied","Data":"e4c5b9081e123dd641ca70034d1eca0a6b238334082ab481b6e34e2bad931817"}
May 28 16:26:45.533338 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:45.533284    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4" event={"ID":"9b4fcbba-c7d2-459b-a8a9-243fee5d691e","Type":"ContainerStarted","Data":"92a99c1952cc99efef0b680553fa87587f41a050948c9398de38d5150ca77894"}
May 28 16:26:50.546915 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:50.546876    2559 generic.go:358] "Generic (PLEG): container finished" podID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerID="92a99c1952cc99efef0b680553fa87587f41a050948c9398de38d5150ca77894" exitCode=0
May 28 16:26:50.547292 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:50.546937    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4" event={"ID":"9b4fcbba-c7d2-459b-a8a9-243fee5d691e","Type":"ContainerDied","Data":"92a99c1952cc99efef0b680553fa87587f41a050948c9398de38d5150ca77894"}
May 28 16:26:51.553223 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:51.553185    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4" event={"ID":"9b4fcbba-c7d2-459b-a8a9-243fee5d691e","Type":"ContainerStarted","Data":"7787ccbb759bf1f3095a1930fa264b74f24ecb83c4533fe2f65bb3e2718bbd91"}
May 28 16:26:51.553730 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:51.553236    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4" event={"ID":"9b4fcbba-c7d2-459b-a8a9-243fee5d691e","Type":"ContainerStarted","Data":"15326a716491270538d4dfc9fd4fd6ac0a1fdbb030b8e8d860c4a4032a295b4b"}
May 28 16:26:51.553730 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:51.553253    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4" event={"ID":"9b4fcbba-c7d2-459b-a8a9-243fee5d691e","Type":"ContainerStarted","Data":"d8c61643bafa9f0c1da70df74950f8b636aa8fb6decbabf8f4e5e82bf53dc2ac"}
May 28 16:26:51.576348 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:51.576261    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4" podStartSLOduration=13.576240128 podStartE2EDuration="13.576240128s" podCreationTimestamp="2026-05-28 16:26:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:26:51.574083956 +0000 UTC m=+1660.133727456" watchObservedRunningTime="2026-05-28 16:26:51.576240128 +0000 UTC m=+1660.135883628"
May 28 16:26:54.287107 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:54.287067    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:54.287515 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:54.287142    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:54.287515 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:54.287156    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:54.288978 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:54.288950    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4" podUID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.21:8816: connect: connection refused"
May 28 16:26:54.289058 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:54.288992    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4" podUID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerName="online" probeResult="failure" output="Get \"https://10.132.0.21:6567/health\": dial tcp 10.132.0.21:6567: connect: connection refused"
May 28 16:26:54.289058 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:54.288995    2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4" podUID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.21:8443: connect: connection refused"
May 28 16:26:57.287928 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:57.287895    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:57.288379 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:57.288066    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:57.288379 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:57.288357    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:57.288537 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:57.288389    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:57.288634 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:57.288618    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:57.288822 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:57.288810    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:57.292039 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:57.292019    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:57.578589 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:57.578505    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:26:57.581461 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:26:57.581437    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:27:59.350372 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:27:59.350270    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"]
May 28 16:27:59.350888 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:27:59.350715    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4" podUID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerName="online" containerID="cri-o://d8c61643bafa9f0c1da70df74950f8b636aa8fb6decbabf8f4e5e82bf53dc2ac" gracePeriod=30
May 28 16:27:59.350888 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:27:59.350763    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4" podUID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerName="ui" containerID="cri-o://7787ccbb759bf1f3095a1930fa264b74f24ecb83c4533fe2f65bb3e2718bbd91" gracePeriod=30
May 28 16:27:59.350888 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:27:59.350782    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4" podUID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerName="offline" containerID="cri-o://15326a716491270538d4dfc9fd4fd6ac0a1fdbb030b8e8d860c4a4032a295b4b" gracePeriod=30
May 28 16:27:59.505592 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:27:59.505555    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"]
May 28 16:27:59.506123 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:27:59.505997    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="registry" containerID="cri-o://7b1f6cf77e608d1c7061f907bf86eb133a002ca6a261436bbec0ec9744928dbe" gracePeriod=30
May 28 16:27:59.506123 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:27:59.506068    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="offline" containerID="cri-o://9e3f2b322e89634a90cc56e813adcb2461833323d00ef373420b0303730177f0" gracePeriod=30
May 28 16:27:59.506123 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:27:59.506081    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="online" containerID="cri-o://90edb2fbec7b87e97a9149749f4dfda0034428af4d47d24bd80f618531aae12c" gracePeriod=30
May 28 16:27:59.506386 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:27:59.506088    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="ui" containerID="cri-o://658a5d76605edbf76e63cc1a1b98bf04dacac956f32acf308b7908ee61bb7a8e" gracePeriod=30
May 28 16:28:00.748979 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:00.748946    2559 generic.go:358] "Generic (PLEG): container finished" podID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerID="658a5d76605edbf76e63cc1a1b98bf04dacac956f32acf308b7908ee61bb7a8e" exitCode=0
May 28 16:28:00.748979 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:00.748970    2559 generic.go:358] "Generic (PLEG): container finished" podID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerID="90edb2fbec7b87e97a9149749f4dfda0034428af4d47d24bd80f618531aae12c" exitCode=0
May 28 16:28:00.749479 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:00.749017    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" event={"ID":"0855f896-0bf9-4db8-8348-b0cf983ddb3f","Type":"ContainerDied","Data":"658a5d76605edbf76e63cc1a1b98bf04dacac956f32acf308b7908ee61bb7a8e"}
May 28 16:28:00.749479 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:00.749053    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" event={"ID":"0855f896-0bf9-4db8-8348-b0cf983ddb3f","Type":"ContainerDied","Data":"90edb2fbec7b87e97a9149749f4dfda0034428af4d47d24bd80f618531aae12c"}
May 28 16:28:00.751084 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:00.751061    2559 generic.go:358] "Generic (PLEG): container finished" podID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerID="7787ccbb759bf1f3095a1930fa264b74f24ecb83c4533fe2f65bb3e2718bbd91" exitCode=0
May 28 16:28:00.751084 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:00.751080    2559 generic.go:358] "Generic (PLEG): container finished" podID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerID="d8c61643bafa9f0c1da70df74950f8b636aa8fb6decbabf8f4e5e82bf53dc2ac" exitCode=0
May 28 16:28:00.751240 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:00.751105    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4" event={"ID":"9b4fcbba-c7d2-459b-a8a9-243fee5d691e","Type":"ContainerDied","Data":"7787ccbb759bf1f3095a1930fa264b74f24ecb83c4533fe2f65bb3e2718bbd91"}
May 28 16:28:00.751240 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:00.751129    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4" event={"ID":"9b4fcbba-c7d2-459b-a8a9-243fee5d691e","Type":"ContainerDied","Data":"d8c61643bafa9f0c1da70df74950f8b636aa8fb6decbabf8f4e5e82bf53dc2ac"}
May 28 16:28:05.754030 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:05.753983    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.20:8443: connect: connection refused"
May 28 16:28:06.341731 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:06.341687    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="online" probeResult="failure" output="Get \"https://10.132.0.20:6567/health\": dial tcp 10.132.0.20:6567: connect: connection refused"
May 28 16:28:07.289100 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:07.289038    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4" podUID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.21:8443: connect: connection refused"
May 28 16:28:07.578793 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:07.578698    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4" podUID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerName="online" probeResult="failure" output="Get \"https://10.132.0.21:6567/health\": dial tcp 10.132.0.21:6567: connect: connection refused"
May 28 16:28:15.754085 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:15.754037    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.20:8443: connect: connection refused"
May 28 16:28:16.341654 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:16.341600    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="online" probeResult="failure" output="Get \"https://10.132.0.20:6567/health\": dial tcp 10.132.0.20:6567: connect: connection refused"
May 28 16:28:17.288642 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:17.288592    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4" podUID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.21:8443: connect: connection refused"
May 28 16:28:17.579658 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:17.579566    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4" podUID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerName="online" probeResult="failure" output="Get \"https://10.132.0.21:6567/health\": dial tcp 10.132.0.21:6567: connect: connection refused"
May 28 16:28:25.753585 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:25.753529    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.20:8443: connect: connection refused"
May 28 16:28:25.754020 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:25.753697    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:28:26.341885 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:26.341842    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="online" probeResult="failure" output="Get \"https://10.132.0.20:6567/health\": dial tcp 10.132.0.20:6567: connect: connection refused"
May 28 16:28:26.342082 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:26.341972    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:28:27.289220 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:27.289172    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4" podUID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.21:8443: connect: connection refused"
May 28 16:28:27.289658 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:27.289313    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:28:27.579651 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:27.579558    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4" podUID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerName="online" probeResult="failure" output="Get \"https://10.132.0.21:6567/health\": dial tcp 10.132.0.21:6567: connect: connection refused"
May 28 16:28:27.579835 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:27.579685    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:28:29.529974 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.529948    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:28:29.660144 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.660040    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-feast-data\") pod \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\" (UID: \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\") "
May 28 16:28:29.660144 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.660106    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2mf6\" (UniqueName: \"kubernetes.io/projected/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-kube-api-access-s2mf6\") pod \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\" (UID: \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\") "
May 28 16:28:29.660424 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.660147    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-registry-tls\") pod \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\" (UID: \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\") "
May 28 16:28:29.660424 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.660178    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-offline-tls\") pod \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\" (UID: \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\") "
May 28 16:28:29.660424 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.660203    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-online-tls\") pod \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\" (UID: \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\") "
May 28 16:28:29.660424 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.660240    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-ui-tls\") pod \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\" (UID: \"9b4fcbba-c7d2-459b-a8a9-243fee5d691e\") "
May 28 16:28:29.660637 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.660620    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9b4fcbba-c7d2-459b-a8a9-243fee5d691e" (UID: "9b4fcbba-c7d2-459b-a8a9-243fee5d691e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 16:28:29.660780 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.660734    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-feast-data" (OuterVolumeSpecName: "feast-data") pod "9b4fcbba-c7d2-459b-a8a9-243fee5d691e" (UID: "9b4fcbba-c7d2-459b-a8a9-243fee5d691e"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 16:28:29.666445 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.666406    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-kube-api-access-s2mf6" (OuterVolumeSpecName: "kube-api-access-s2mf6") pod "9b4fcbba-c7d2-459b-a8a9-243fee5d691e" (UID: "9b4fcbba-c7d2-459b-a8a9-243fee5d691e"). InnerVolumeSpecName "kube-api-access-s2mf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 16:28:29.666609 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.666425    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "9b4fcbba-c7d2-459b-a8a9-243fee5d691e" (UID: "9b4fcbba-c7d2-459b-a8a9-243fee5d691e"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:28:29.666878 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.666854    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-online-tls" (OuterVolumeSpecName: "online-tls") pod "9b4fcbba-c7d2-459b-a8a9-243fee5d691e" (UID: "9b4fcbba-c7d2-459b-a8a9-243fee5d691e"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:28:29.666957 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.666876    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "9b4fcbba-c7d2-459b-a8a9-243fee5d691e" (UID: "9b4fcbba-c7d2-459b-a8a9-243fee5d691e"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:28:29.761427 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.761388    2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-registry-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:28:29.761427 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.761426    2559 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-offline-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:28:29.761427 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.761435    2559 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-online-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:28:29.761663 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.761444    2559 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-ui-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:28:29.761663 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.761452    2559 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-feast-data\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:28:29.761663 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.761468    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2mf6\" (UniqueName: \"kubernetes.io/projected/9b4fcbba-c7d2-459b-a8a9-243fee5d691e-kube-api-access-s2mf6\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:28:29.829275 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.829241    2559 generic.go:358] "Generic (PLEG): container finished" podID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerID="9e3f2b322e89634a90cc56e813adcb2461833323d00ef373420b0303730177f0" exitCode=137
May 28 16:28:29.829275 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.829268    2559 generic.go:358] "Generic (PLEG): container finished" podID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerID="7b1f6cf77e608d1c7061f907bf86eb133a002ca6a261436bbec0ec9744928dbe" exitCode=137
May 28 16:28:29.829499 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.829301    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" event={"ID":"0855f896-0bf9-4db8-8348-b0cf983ddb3f","Type":"ContainerDied","Data":"9e3f2b322e89634a90cc56e813adcb2461833323d00ef373420b0303730177f0"}
May 28 16:28:29.829499 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.829349    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" event={"ID":"0855f896-0bf9-4db8-8348-b0cf983ddb3f","Type":"ContainerDied","Data":"7b1f6cf77e608d1c7061f907bf86eb133a002ca6a261436bbec0ec9744928dbe"}
May 28 16:28:29.831215 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.831188    2559 generic.go:358] "Generic (PLEG): container finished" podID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerID="15326a716491270538d4dfc9fd4fd6ac0a1fdbb030b8e8d860c4a4032a295b4b" exitCode=137
May 28 16:28:29.831299 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.831233    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4" event={"ID":"9b4fcbba-c7d2-459b-a8a9-243fee5d691e","Type":"ContainerDied","Data":"15326a716491270538d4dfc9fd4fd6ac0a1fdbb030b8e8d860c4a4032a295b4b"}
May 28 16:28:29.831299 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.831263    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4" event={"ID":"9b4fcbba-c7d2-459b-a8a9-243fee5d691e","Type":"ContainerDied","Data":"891c8394820bc30e2d0477c7ee7e5602338bdc9a1a163e512c9f9c1812fc0a79"}
May 28 16:28:29.831299 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.831277    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"
May 28 16:28:29.831299 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.831284    2559 scope.go:117] "RemoveContainer" containerID="7787ccbb759bf1f3095a1930fa264b74f24ecb83c4533fe2f65bb3e2718bbd91"
May 28 16:28:29.840017 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.839997    2559 scope.go:117] "RemoveContainer" containerID="15326a716491270538d4dfc9fd4fd6ac0a1fdbb030b8e8d860c4a4032a295b4b"
May 28 16:28:29.846699 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.846679    2559 scope.go:117] "RemoveContainer" containerID="d8c61643bafa9f0c1da70df74950f8b636aa8fb6decbabf8f4e5e82bf53dc2ac"
May 28 16:28:29.854037 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.854010    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"]
May 28 16:28:29.854395 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.854375    2559 scope.go:117] "RemoveContainer" containerID="92a99c1952cc99efef0b680553fa87587f41a050948c9398de38d5150ca77894"
May 28 16:28:29.857143 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.857115    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-njgr4"]
May 28 16:28:29.862079 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.862051    2559 scope.go:117] "RemoveContainer" containerID="e4c5b9081e123dd641ca70034d1eca0a6b238334082ab481b6e34e2bad931817"
May 28 16:28:29.868396 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.868379    2559 scope.go:117] "RemoveContainer" containerID="7787ccbb759bf1f3095a1930fa264b74f24ecb83c4533fe2f65bb3e2718bbd91"
May 28 16:28:29.868633 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:28:29.868615    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7787ccbb759bf1f3095a1930fa264b74f24ecb83c4533fe2f65bb3e2718bbd91\": container with ID starting with 7787ccbb759bf1f3095a1930fa264b74f24ecb83c4533fe2f65bb3e2718bbd91 not found: ID does not exist" containerID="7787ccbb759bf1f3095a1930fa264b74f24ecb83c4533fe2f65bb3e2718bbd91"
May 28 16:28:29.868678 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.868642    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7787ccbb759bf1f3095a1930fa264b74f24ecb83c4533fe2f65bb3e2718bbd91"} err="failed to get container status \"7787ccbb759bf1f3095a1930fa264b74f24ecb83c4533fe2f65bb3e2718bbd91\": rpc error: code = NotFound desc = could not find container \"7787ccbb759bf1f3095a1930fa264b74f24ecb83c4533fe2f65bb3e2718bbd91\": container with ID starting with 7787ccbb759bf1f3095a1930fa264b74f24ecb83c4533fe2f65bb3e2718bbd91 not found: ID does not exist"
May 28 16:28:29.868678 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.868663    2559 scope.go:117] "RemoveContainer" containerID="15326a716491270538d4dfc9fd4fd6ac0a1fdbb030b8e8d860c4a4032a295b4b"
May 28 16:28:29.868891 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:28:29.868875    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15326a716491270538d4dfc9fd4fd6ac0a1fdbb030b8e8d860c4a4032a295b4b\": container with ID starting with 15326a716491270538d4dfc9fd4fd6ac0a1fdbb030b8e8d860c4a4032a295b4b not found: ID does not exist" containerID="15326a716491270538d4dfc9fd4fd6ac0a1fdbb030b8e8d860c4a4032a295b4b"
May 28 16:28:29.868936 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.868896    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15326a716491270538d4dfc9fd4fd6ac0a1fdbb030b8e8d860c4a4032a295b4b"} err="failed to get container status \"15326a716491270538d4dfc9fd4fd6ac0a1fdbb030b8e8d860c4a4032a295b4b\": rpc error: code = NotFound desc = could not find container \"15326a716491270538d4dfc9fd4fd6ac0a1fdbb030b8e8d860c4a4032a295b4b\": container with ID starting with 15326a716491270538d4dfc9fd4fd6ac0a1fdbb030b8e8d860c4a4032a295b4b not found: ID does not exist"
May 28 16:28:29.868936 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.868911    2559 scope.go:117] "RemoveContainer" containerID="d8c61643bafa9f0c1da70df74950f8b636aa8fb6decbabf8f4e5e82bf53dc2ac"
May 28 16:28:29.869142 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:28:29.869125    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8c61643bafa9f0c1da70df74950f8b636aa8fb6decbabf8f4e5e82bf53dc2ac\": container with ID starting with d8c61643bafa9f0c1da70df74950f8b636aa8fb6decbabf8f4e5e82bf53dc2ac not found: ID does not exist" containerID="d8c61643bafa9f0c1da70df74950f8b636aa8fb6decbabf8f4e5e82bf53dc2ac"
May 28 16:28:29.869183 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.869148    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8c61643bafa9f0c1da70df74950f8b636aa8fb6decbabf8f4e5e82bf53dc2ac"} err="failed to get container status \"d8c61643bafa9f0c1da70df74950f8b636aa8fb6decbabf8f4e5e82bf53dc2ac\": rpc error: code = NotFound desc = could not find container \"d8c61643bafa9f0c1da70df74950f8b636aa8fb6decbabf8f4e5e82bf53dc2ac\": container with ID starting with d8c61643bafa9f0c1da70df74950f8b636aa8fb6decbabf8f4e5e82bf53dc2ac not found: ID does not exist"
May 28 16:28:29.869183 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.869163    2559 scope.go:117] "RemoveContainer" containerID="92a99c1952cc99efef0b680553fa87587f41a050948c9398de38d5150ca77894"
May 28 16:28:29.869365 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:28:29.869349    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92a99c1952cc99efef0b680553fa87587f41a050948c9398de38d5150ca77894\": container with ID starting with 92a99c1952cc99efef0b680553fa87587f41a050948c9398de38d5150ca77894 not found: ID does not exist" containerID="92a99c1952cc99efef0b680553fa87587f41a050948c9398de38d5150ca77894"
May 28 16:28:29.869403 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.869371    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92a99c1952cc99efef0b680553fa87587f41a050948c9398de38d5150ca77894"} err="failed to get container status \"92a99c1952cc99efef0b680553fa87587f41a050948c9398de38d5150ca77894\": rpc error: code = NotFound desc = could not find container \"92a99c1952cc99efef0b680553fa87587f41a050948c9398de38d5150ca77894\": container with ID starting with 92a99c1952cc99efef0b680553fa87587f41a050948c9398de38d5150ca77894 not found: ID does not exist"
May 28 16:28:29.869403 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.869385    2559 scope.go:117] "RemoveContainer" containerID="e4c5b9081e123dd641ca70034d1eca0a6b238334082ab481b6e34e2bad931817"
May 28 16:28:29.869600 ip-10-0-133-120 kubenswrapper[2559]: E0528 16:28:29.869583    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4c5b9081e123dd641ca70034d1eca0a6b238334082ab481b6e34e2bad931817\": container with ID starting with e4c5b9081e123dd641ca70034d1eca0a6b238334082ab481b6e34e2bad931817 not found: ID does not exist" containerID="e4c5b9081e123dd641ca70034d1eca0a6b238334082ab481b6e34e2bad931817"
May 28 16:28:29.869653 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:29.869609    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4c5b9081e123dd641ca70034d1eca0a6b238334082ab481b6e34e2bad931817"} err="failed to get container status \"e4c5b9081e123dd641ca70034d1eca0a6b238334082ab481b6e34e2bad931817\": rpc error: code = NotFound desc = could not find container \"e4c5b9081e123dd641ca70034d1eca0a6b238334082ab481b6e34e2bad931817\": container with ID starting with e4c5b9081e123dd641ca70034d1eca0a6b238334082ab481b6e34e2bad931817 not found: ID does not exist"
May 28 16:28:30.073788 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.073749    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" path="/var/lib/kubelet/pods/9b4fcbba-c7d2-459b-a8a9-243fee5d691e/volumes"
May 28 16:28:30.151183 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.151155    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:28:30.265755 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.265721    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-ui-tls\") pod \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\" (UID: \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\") "
May 28 16:28:30.265755 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.265767    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-registry-tls\") pod \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\" (UID: \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\") "
May 28 16:28:30.266114 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.265787    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-online-tls\") pod \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\" (UID: \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\") "
May 28 16:28:30.266114 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.265812    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmvt6\" (UniqueName: \"kubernetes.io/projected/0855f896-0bf9-4db8-8348-b0cf983ddb3f-kube-api-access-vmvt6\") pod \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\" (UID: \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\") "
May 28 16:28:30.266114 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.265839    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0855f896-0bf9-4db8-8348-b0cf983ddb3f-feast-data\") pod \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\" (UID: \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\") "
May 28 16:28:30.266114 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.265853    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-offline-tls\") pod \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\" (UID: \"0855f896-0bf9-4db8-8348-b0cf983ddb3f\") "
May 28 16:28:30.266469 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.266436    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0855f896-0bf9-4db8-8348-b0cf983ddb3f-feast-data" (OuterVolumeSpecName: "feast-data") pod "0855f896-0bf9-4db8-8348-b0cf983ddb3f" (UID: "0855f896-0bf9-4db8-8348-b0cf983ddb3f"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 16:28:30.268045 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.268019    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "0855f896-0bf9-4db8-8348-b0cf983ddb3f" (UID: "0855f896-0bf9-4db8-8348-b0cf983ddb3f"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:28:30.268143 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.268048    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "0855f896-0bf9-4db8-8348-b0cf983ddb3f" (UID: "0855f896-0bf9-4db8-8348-b0cf983ddb3f"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:28:30.268143 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.268061    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-online-tls" (OuterVolumeSpecName: "online-tls") pod "0855f896-0bf9-4db8-8348-b0cf983ddb3f" (UID: "0855f896-0bf9-4db8-8348-b0cf983ddb3f"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:28:30.268297 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.268282    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0855f896-0bf9-4db8-8348-b0cf983ddb3f-kube-api-access-vmvt6" (OuterVolumeSpecName: "kube-api-access-vmvt6") pod "0855f896-0bf9-4db8-8348-b0cf983ddb3f" (UID: "0855f896-0bf9-4db8-8348-b0cf983ddb3f"). InnerVolumeSpecName "kube-api-access-vmvt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 16:28:30.268297 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.268288    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0855f896-0bf9-4db8-8348-b0cf983ddb3f" (UID: "0855f896-0bf9-4db8-8348-b0cf983ddb3f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 16:28:30.366863 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.366792    2559 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-ui-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:28:30.366863 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.366820    2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-registry-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:28:30.366863 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.366830    2559 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-online-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:28:30.366863 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.366839    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmvt6\" (UniqueName: \"kubernetes.io/projected/0855f896-0bf9-4db8-8348-b0cf983ddb3f-kube-api-access-vmvt6\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:28:30.366863 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.366847    2559 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0855f896-0bf9-4db8-8348-b0cf983ddb3f-feast-data\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:28:30.366863 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.366856    2559 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0855f896-0bf9-4db8-8348-b0cf983ddb3f-offline-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
May 28 16:28:30.836026 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.835992    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk" event={"ID":"0855f896-0bf9-4db8-8348-b0cf983ddb3f","Type":"ContainerDied","Data":"15971ca515d07dc59dca0838609db8e0387bd97005c541af6153d050563d9bc8"}
May 28 16:28:30.836442 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.836005    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"
May 28 16:28:30.836442 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.836045    2559 scope.go:117] "RemoveContainer" containerID="658a5d76605edbf76e63cc1a1b98bf04dacac956f32acf308b7908ee61bb7a8e"
May 28 16:28:30.844484 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.844465    2559 scope.go:117] "RemoveContainer" containerID="9e3f2b322e89634a90cc56e813adcb2461833323d00ef373420b0303730177f0"
May 28 16:28:30.851195 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.851176    2559 scope.go:117] "RemoveContainer" containerID="90edb2fbec7b87e97a9149749f4dfda0034428af4d47d24bd80f618531aae12c"
May 28 16:28:30.856836 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.856810    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"]
May 28 16:28:30.857919 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.857904    2559 scope.go:117] "RemoveContainer" containerID="7b1f6cf77e608d1c7061f907bf86eb133a002ca6a261436bbec0ec9744928dbe"
May 28 16:28:30.861418 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.861392    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-t5pdk"]
May 28 16:28:30.865206 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.865189    2559 scope.go:117] "RemoveContainer" containerID="5fabe2a303bd1f562b66e1707361aa8a3fa3088f489dc0ab9f78ab7c197360cd"
May 28 16:28:30.871542 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:30.871526    2559 scope.go:117] "RemoveContainer" containerID="1d47c0a3e9c78e990e1bbdf81df57b260570fa97e50abaa4ea72aa29cba58172"
May 28 16:28:32.073876 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:28:32.073846    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" path="/var/lib/kubelet/pods/0855f896-0bf9-4db8-8348-b0cf983ddb3f/volumes"
May 28 16:29:05.747765 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.747684    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-r7qnr/must-gather-wbnkg"]
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.747905    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="ui"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.747916    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="ui"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.747928    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerName="offline"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.747934    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerName="offline"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.747942    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerName="feast-apply"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.747948    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerName="feast-apply"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.747954    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="feast-init"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.747959    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="feast-init"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.747965    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="feast-apply"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.747970    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="feast-apply"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.747978    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="online"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.747982    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="online"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.747990    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="registry"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.747996    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="registry"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.748002    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="offline"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.748007    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="offline"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.748013    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerName="online"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.748017    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerName="online"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.748022    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerName="ui"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.748027    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerName="ui"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.748035    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerName="feast-init"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.748040    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerName="feast-init"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.748076    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerName="online"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.748083    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerName="ui"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.748089    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="9b4fcbba-c7d2-459b-a8a9-243fee5d691e" containerName="offline"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.748094    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="registry"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.748099    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="online"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.748106    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="offline"
May 28 16:29:05.748209 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.748111    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="0855f896-0bf9-4db8-8348-b0cf983ddb3f" containerName="ui"
May 28 16:29:05.751221 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.751203    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-r7qnr/must-gather-wbnkg"
May 28 16:29:05.753908 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.753887    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-r7qnr\"/\"default-dockercfg-j8rbg\""
May 28 16:29:05.753908 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.753905    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-r7qnr\"/\"openshift-service-ca.crt\""
May 28 16:29:05.754298 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.754282    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-r7qnr\"/\"kube-root-ca.crt\""
May 28 16:29:05.761896 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.761878    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-r7qnr/must-gather-wbnkg"]
May 28 16:29:05.919835 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.919798    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hp4c\" (UniqueName: \"kubernetes.io/projected/8acf8775-ddb9-4575-88d3-a40afdb35ce3-kube-api-access-9hp4c\") pod \"must-gather-wbnkg\" (UID: \"8acf8775-ddb9-4575-88d3-a40afdb35ce3\") " pod="openshift-must-gather-r7qnr/must-gather-wbnkg"
May 28 16:29:05.920007 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:05.919852    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8acf8775-ddb9-4575-88d3-a40afdb35ce3-must-gather-output\") pod \"must-gather-wbnkg\" (UID: \"8acf8775-ddb9-4575-88d3-a40afdb35ce3\") " pod="openshift-must-gather-r7qnr/must-gather-wbnkg"
May 28 16:29:06.020663 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:06.020565    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8acf8775-ddb9-4575-88d3-a40afdb35ce3-must-gather-output\") pod \"must-gather-wbnkg\" (UID: \"8acf8775-ddb9-4575-88d3-a40afdb35ce3\") " pod="openshift-must-gather-r7qnr/must-gather-wbnkg"
May 28 16:29:06.020663 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:06.020624    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9hp4c\" (UniqueName: \"kubernetes.io/projected/8acf8775-ddb9-4575-88d3-a40afdb35ce3-kube-api-access-9hp4c\") pod \"must-gather-wbnkg\" (UID: \"8acf8775-ddb9-4575-88d3-a40afdb35ce3\") " pod="openshift-must-gather-r7qnr/must-gather-wbnkg"
May 28 16:29:06.020946 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:06.020925    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8acf8775-ddb9-4575-88d3-a40afdb35ce3-must-gather-output\") pod \"must-gather-wbnkg\" (UID: \"8acf8775-ddb9-4575-88d3-a40afdb35ce3\") " pod="openshift-must-gather-r7qnr/must-gather-wbnkg"
May 28 16:29:06.029355 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:06.029306    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hp4c\" (UniqueName: \"kubernetes.io/projected/8acf8775-ddb9-4575-88d3-a40afdb35ce3-kube-api-access-9hp4c\") pod \"must-gather-wbnkg\" (UID: \"8acf8775-ddb9-4575-88d3-a40afdb35ce3\") " pod="openshift-must-gather-r7qnr/must-gather-wbnkg"
May 28 16:29:06.060443 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:06.060403    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-r7qnr/must-gather-wbnkg"
May 28 16:29:06.183663 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:06.183631    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-r7qnr/must-gather-wbnkg"]
May 28 16:29:06.187091 ip-10-0-133-120 kubenswrapper[2559]: W0528 16:29:06.187062    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8acf8775_ddb9_4575_88d3_a40afdb35ce3.slice/crio-5eea965329665e1acad695e27646658acf679cff03424464475dba76830be39c WatchSource:0}: Error finding container 5eea965329665e1acad695e27646658acf679cff03424464475dba76830be39c: Status 404 returned error can't find the container with id 5eea965329665e1acad695e27646658acf679cff03424464475dba76830be39c
May 28 16:29:06.189176 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:06.189162    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 16:29:06.927842 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:06.927808    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-r7qnr/must-gather-wbnkg" event={"ID":"8acf8775-ddb9-4575-88d3-a40afdb35ce3","Type":"ContainerStarted","Data":"5eea965329665e1acad695e27646658acf679cff03424464475dba76830be39c"}
May 28 16:29:07.932058 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:07.932016    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-r7qnr/must-gather-wbnkg" event={"ID":"8acf8775-ddb9-4575-88d3-a40afdb35ce3","Type":"ContainerStarted","Data":"6647eda0b1b053e06f322ece5d8b476b5fdec9389e9eee3c740061a31cde4865"}
May 28 16:29:07.932058 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:07.932058    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-r7qnr/must-gather-wbnkg" event={"ID":"8acf8775-ddb9-4575-88d3-a40afdb35ce3","Type":"ContainerStarted","Data":"2564a739e33dfb6cb29c9e7de706e1a40b28b43f98a535f84e3f9b93f7baee46"}
May 28 16:29:07.947982 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:07.947934    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-r7qnr/must-gather-wbnkg" podStartSLOduration=2.161240216 podStartE2EDuration="2.94791812s" podCreationTimestamp="2026-05-28 16:29:05 +0000 UTC" firstStartedPulling="2026-05-28 16:29:06.189308348 +0000 UTC m=+1794.748951824" lastFinishedPulling="2026-05-28 16:29:06.975986251 +0000 UTC m=+1795.535629728" observedRunningTime="2026-05-28 16:29:07.946748477 +0000 UTC m=+1796.506391976" watchObservedRunningTime="2026-05-28 16:29:07.94791812 +0000 UTC m=+1796.507561617"
May 28 16:29:08.269396 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:08.269364    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bkvg9_369ece78-a49b-484b-8521-d1f4656e1c6e/global-pull-secret-syncer/0.log"
May 28 16:29:08.480155 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:08.480124    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vj78z_2c3786be-4e2a-4969-8c45-45c3b356bc48/konnectivity-agent/0.log"
May 28 16:29:08.524989 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:08.524908    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-120.ec2.internal_6db7b9c4232c4daf9a5a539bc775452b/haproxy/0.log"
May 28 16:29:11.970908 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:11.970845    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-58nm8_a7fda48a-22aa-4dc4-a60d-ecf919ffc212/node-exporter/0.log"
May 28 16:29:11.994861 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:11.994822    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-58nm8_a7fda48a-22aa-4dc4-a60d-ecf919ffc212/kube-rbac-proxy/0.log"
May 28 16:29:12.016309 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:12.016277    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-58nm8_a7fda48a-22aa-4dc4-a60d-ecf919ffc212/init-textfile/0.log"
May 28 16:29:15.012171 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.012133    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-r7qnr/perf-node-gather-daemonset-8zkcn"]
May 28 16:29:15.019516 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.019489    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-8zkcn"
May 28 16:29:15.025023 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.024997    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-r7qnr/perf-node-gather-daemonset-8zkcn"]
May 28 16:29:15.095189 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.095137    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/519ba6a6-0275-4ad0-b8a3-6d825f2c24f3-podres\") pod \"perf-node-gather-daemonset-8zkcn\" (UID: \"519ba6a6-0275-4ad0-b8a3-6d825f2c24f3\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-8zkcn"
May 28 16:29:15.095410 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.095196    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/519ba6a6-0275-4ad0-b8a3-6d825f2c24f3-proc\") pod \"perf-node-gather-daemonset-8zkcn\" (UID: \"519ba6a6-0275-4ad0-b8a3-6d825f2c24f3\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-8zkcn"
May 28 16:29:15.095410 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.095265    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxg74\" (UniqueName: \"kubernetes.io/projected/519ba6a6-0275-4ad0-b8a3-6d825f2c24f3-kube-api-access-qxg74\") pod \"perf-node-gather-daemonset-8zkcn\" (UID: \"519ba6a6-0275-4ad0-b8a3-6d825f2c24f3\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-8zkcn"
May 28 16:29:15.095410 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.095297    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/519ba6a6-0275-4ad0-b8a3-6d825f2c24f3-sys\") pod \"perf-node-gather-daemonset-8zkcn\" (UID: \"519ba6a6-0275-4ad0-b8a3-6d825f2c24f3\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-8zkcn"
May 28 16:29:15.095410 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.095338    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/519ba6a6-0275-4ad0-b8a3-6d825f2c24f3-lib-modules\") pod \"perf-node-gather-daemonset-8zkcn\" (UID: \"519ba6a6-0275-4ad0-b8a3-6d825f2c24f3\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-8zkcn"
May 28 16:29:15.196579 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.196536    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qxg74\" (UniqueName: \"kubernetes.io/projected/519ba6a6-0275-4ad0-b8a3-6d825f2c24f3-kube-api-access-qxg74\") pod \"perf-node-gather-daemonset-8zkcn\" (UID: \"519ba6a6-0275-4ad0-b8a3-6d825f2c24f3\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-8zkcn"
May 28 16:29:15.196763 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.196592    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/519ba6a6-0275-4ad0-b8a3-6d825f2c24f3-sys\") pod \"perf-node-gather-daemonset-8zkcn\" (UID: \"519ba6a6-0275-4ad0-b8a3-6d825f2c24f3\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-8zkcn"
May 28 16:29:15.196763 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.196618    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/519ba6a6-0275-4ad0-b8a3-6d825f2c24f3-lib-modules\") pod \"perf-node-gather-daemonset-8zkcn\" (UID: \"519ba6a6-0275-4ad0-b8a3-6d825f2c24f3\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-8zkcn"
May 28 16:29:15.196763 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.196730    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/519ba6a6-0275-4ad0-b8a3-6d825f2c24f3-sys\") pod \"perf-node-gather-daemonset-8zkcn\" (UID: \"519ba6a6-0275-4ad0-b8a3-6d825f2c24f3\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-8zkcn"
May 28 16:29:15.196929 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.196784    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/519ba6a6-0275-4ad0-b8a3-6d825f2c24f3-podres\") pod \"perf-node-gather-daemonset-8zkcn\" (UID: \"519ba6a6-0275-4ad0-b8a3-6d825f2c24f3\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-8zkcn"
May 28 16:29:15.196929 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.196808    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/519ba6a6-0275-4ad0-b8a3-6d825f2c24f3-lib-modules\") pod \"perf-node-gather-daemonset-8zkcn\" (UID: \"519ba6a6-0275-4ad0-b8a3-6d825f2c24f3\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-8zkcn"
May 28 16:29:15.196929 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.196828    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/519ba6a6-0275-4ad0-b8a3-6d825f2c24f3-proc\") pod \"perf-node-gather-daemonset-8zkcn\" (UID: \"519ba6a6-0275-4ad0-b8a3-6d825f2c24f3\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-8zkcn"
May 28 16:29:15.196929 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.196894    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/519ba6a6-0275-4ad0-b8a3-6d825f2c24f3-podres\") pod \"perf-node-gather-daemonset-8zkcn\" (UID: \"519ba6a6-0275-4ad0-b8a3-6d825f2c24f3\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-8zkcn"
May 28 16:29:15.197054 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.196941    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/519ba6a6-0275-4ad0-b8a3-6d825f2c24f3-proc\") pod \"perf-node-gather-daemonset-8zkcn\" (UID: \"519ba6a6-0275-4ad0-b8a3-6d825f2c24f3\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-8zkcn"
May 28 16:29:15.204979 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.204950    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxg74\" (UniqueName: \"kubernetes.io/projected/519ba6a6-0275-4ad0-b8a3-6d825f2c24f3-kube-api-access-qxg74\") pod \"perf-node-gather-daemonset-8zkcn\" (UID: \"519ba6a6-0275-4ad0-b8a3-6d825f2c24f3\") " pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-8zkcn"
May 28 16:29:15.332715 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.332613    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-8zkcn"
May 28 16:29:15.480698 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.480663    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-r7qnr/perf-node-gather-daemonset-8zkcn"]
May 28 16:29:15.484985 ip-10-0-133-120 kubenswrapper[2559]: W0528 16:29:15.484956    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod519ba6a6_0275_4ad0_b8a3_6d825f2c24f3.slice/crio-34f9ea91bfc670464c189738cdc3e914bd83e0274392b774fac0a53ba48f984b WatchSource:0}: Error finding container 34f9ea91bfc670464c189738cdc3e914bd83e0274392b774fac0a53ba48f984b: Status 404 returned error can't find the container with id 34f9ea91bfc670464c189738cdc3e914bd83e0274392b774fac0a53ba48f984b
May 28 16:29:15.535248 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.535227    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b2vxs_a45a3b0c-89c4-4212-967e-a1b96d0c8a81/dns/0.log"
May 28 16:29:15.555040 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.555017    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b2vxs_a45a3b0c-89c4-4212-967e-a1b96d0c8a81/kube-rbac-proxy/0.log"
May 28 16:29:15.731343 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.731248    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ztbwc_3d8296a6-b347-4d39-a758-d049ee69ef30/dns-node-resolver/0.log"
May 28 16:29:15.959154 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.959119    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-8zkcn" event={"ID":"519ba6a6-0275-4ad0-b8a3-6d825f2c24f3","Type":"ContainerStarted","Data":"bf9ab822656600938abaaaeee2733c8b7364231b6448bb94b9900b82a642ddd6"}
May 28 16:29:15.959341 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.959161    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-8zkcn" event={"ID":"519ba6a6-0275-4ad0-b8a3-6d825f2c24f3","Type":"ContainerStarted","Data":"34f9ea91bfc670464c189738cdc3e914bd83e0274392b774fac0a53ba48f984b"}
May 28 16:29:15.959341 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.959195    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-8zkcn"
May 28 16:29:15.974883 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:15.974841    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-8zkcn" podStartSLOduration=1.974827785 podStartE2EDuration="1.974827785s" podCreationTimestamp="2026-05-28 16:29:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 16:29:15.973616639 +0000 UTC m=+1804.533260139" watchObservedRunningTime="2026-05-28 16:29:15.974827785 +0000 UTC m=+1804.534471284"
May 28 16:29:16.129252 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:16.129224    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5dd8s_fd2dd070-87e3-4cda-8fe0-1635aeb20687/node-ca/0.log"
May 28 16:29:17.093055 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:17.093017    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ddmdp_6123b655-5980-4479-9c2a-829cde441719/serve-healthcheck-canary/0.log"
May 28 16:29:17.587036 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:17.587008    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tfsfh_dab12b45-9da4-47e1-8464-f515ac10e266/kube-rbac-proxy/0.log"
May 28 16:29:17.605397 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:17.605374    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tfsfh_dab12b45-9da4-47e1-8464-f515ac10e266/exporter/0.log"
May 28 16:29:17.624111 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:17.624087    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tfsfh_dab12b45-9da4-47e1-8464-f515ac10e266/extractor/0.log"
May 28 16:29:21.974295 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:21.973594    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-r7qnr/perf-node-gather-daemonset-8zkcn"
May 28 16:29:22.771156 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:22.771124    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fgmnc_bdd26bcc-76e5-4f92-812a-d7d05a31e959/kube-multus-additional-cni-plugins/0.log"
May 28 16:29:22.811058 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:22.811029    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fgmnc_bdd26bcc-76e5-4f92-812a-d7d05a31e959/egress-router-binary-copy/0.log"
May 28 16:29:22.850609 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:22.850580    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fgmnc_bdd26bcc-76e5-4f92-812a-d7d05a31e959/cni-plugins/0.log"
May 28 16:29:22.884084 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:22.884055    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fgmnc_bdd26bcc-76e5-4f92-812a-d7d05a31e959/bond-cni-plugin/0.log"
May 28 16:29:22.926430 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:22.926400    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fgmnc_bdd26bcc-76e5-4f92-812a-d7d05a31e959/routeoverride-cni/0.log"
May 28 16:29:22.966294 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:22.966262    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fgmnc_bdd26bcc-76e5-4f92-812a-d7d05a31e959/whereabouts-cni-bincopy/0.log"
May 28 16:29:22.997529 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:22.997499    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fgmnc_bdd26bcc-76e5-4f92-812a-d7d05a31e959/whereabouts-cni/0.log"
May 28 16:29:23.209143 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:23.209062    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-m8b2m_bc5f0404-5bae-450f-92d9-d1dbcd458e1a/kube-multus/0.log"
May 28 16:29:23.234467 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:23.234436    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-69vjf_bea98d0f-e54c-4da9-a207-f692c8781bea/network-metrics-daemon/0.log"
May 28 16:29:23.262394 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:23.262356    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-69vjf_bea98d0f-e54c-4da9-a207-f692c8781bea/kube-rbac-proxy/0.log"
May 28 16:29:24.441264 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:24.441226    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bgmxp_fd6dd018-b7d1-426f-ae3f-4d10ac882495/ovn-controller/0.log"
May 28 16:29:24.471551 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:24.471518    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bgmxp_fd6dd018-b7d1-426f-ae3f-4d10ac882495/ovn-acl-logging/0.log"
May 28 16:29:24.494512 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:24.494485    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bgmxp_fd6dd018-b7d1-426f-ae3f-4d10ac882495/kube-rbac-proxy-node/0.log"
May 28 16:29:24.519585 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:24.519554    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bgmxp_fd6dd018-b7d1-426f-ae3f-4d10ac882495/kube-rbac-proxy-ovn-metrics/0.log"
May 28 16:29:24.537745 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:24.537706    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bgmxp_fd6dd018-b7d1-426f-ae3f-4d10ac882495/northd/0.log"
May 28 16:29:24.559374 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:24.559312    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bgmxp_fd6dd018-b7d1-426f-ae3f-4d10ac882495/nbdb/0.log"
May 28 16:29:24.583648 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:24.583616    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bgmxp_fd6dd018-b7d1-426f-ae3f-4d10ac882495/sbdb/0.log"
May 28 16:29:24.753735 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:24.753665    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bgmxp_fd6dd018-b7d1-426f-ae3f-4d10ac882495/ovnkube-controller/0.log"
May 28 16:29:25.968789 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:25.968758    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-lw6n8_c73529ee-0f31-4285-85c2-51cdfd913e80/network-check-target-container/0.log"
May 28 16:29:26.841442 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:26.841410    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-8krdd_ef8f4bc2-1970-4531-9488-ec899cdc5f35/iptables-alerter/0.log"
May 28 16:29:27.412866 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:27.412833    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-56thk_263ec2d4-e89b-48d8-9279-d702116092ad/tuned/0.log"
May 28 16:29:30.665169 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:30.665140    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-rjwv8_c4578fd3-b4bc-4b10-85df-d6f40b590795/csi-driver/0.log"
May 28 16:29:30.687622 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:30.687594    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-rjwv8_c4578fd3-b4bc-4b10-85df-d6f40b590795/csi-node-driver-registrar/0.log"
May 28 16:29:30.708572 ip-10-0-133-120 kubenswrapper[2559]: I0528 16:29:30.708538    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-rjwv8_c4578fd3-b4bc-4b10-85df-d6f40b590795/csi-liveness-probe/0.log"